{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 02_Train_TransE_l1\n",
    "#\n",
    "# created by LuYF-Lemon-love <luyanfeng_nlp@qq.com> on February 26, 2023\n",
    "# updated by LuYF-Lemon-love <luyanfeng_nlp@qq.com> on February 26, 2023\n",
    "#\n",
    "# 该脚本展示了如何在 DRKG 上训练模型 (TransE_l1), 并利用网格搜索寻找到最优参数.\n",
    "#\n",
    "# 需要的包:\n",
    "#          torch\n",
    "#          dgl, version: 0.4.3\n",
    "#          dglke\n",
    "#          numpy\n",
    "#\n",
    "# 需要的文件:\n",
    "#          ./dataset\n",
    "#\n",
    "# 源教程链接: https://github.com/gnn4dr/DRKG/blob/master/embedding_analysis/Train_embeddings.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training DRKG Using TransE_L1\n",
    "\n",
    "这个 notebook 展示了如何在 DRKG 上训练模型 (TransE_l1), 并利用网格搜索寻找到最优参数."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入需要的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 网格搜索参数\n",
    "\n",
    "我们能使用 DGL-KE 命令训练 TransE_l1 模型, 关于如何使用 DGL-KE 的更多信息请参考 https://github.com/awslabs/dgl-ke.\n",
    "\n",
    "这里我们使用两个 GPU 训练模型."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: **6**, 12, 18\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.121 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.39356343003958466\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.44522365992218255\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.39417278753221036\n",
      "[proc 1][Train](20000/100000) average loss: 0.41939354507625104\n",
      "[proc 1][Train](20000/100000) average regularization: 3.0065776003607426e-05\n",
      "[proc 1][Train] 20000 steps take 756.932 seconds\n",
      "[proc 1]sample: 63.101, forward: 336.245, backward: 51.322, update: 305.905\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4452695511266589\n",
      "[proc 0][Train](20000/100000) average loss: 0.4197211692348123\n",
      "[proc 0][Train](20000/100000) average regularization: 3.007778224471167e-05\n",
      "[proc 0][Train] 20000 steps take 756.961 seconds\n",
      "[proc 0]sample: 62.209, forward: 323.768, backward: 48.603, update: 305.676\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3987347657456994\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4465108074888587\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.39897115109711884\n",
      "[proc 1][Train](40000/100000) average loss: 0.4226227866798639\n",
      "[proc 1][Train](40000/100000) average regularization: 3.557294392612675e-05\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.44659083904623986\n",
      "[proc 1][Train] 20000 steps take 757.303 seconds\n",
      "[proc 1]sample: 63.542, forward: 335.049, backward: 51.699, update: 306.737\n",
      "[proc 0][Train](40000/100000) average loss: 0.42278099519312384\n",
      "[proc 0][Train](40000/100000) average regularization: 3.560002553713275e-05\n",
      "[proc 0][Train] 20000 steps take 757.293 seconds\n",
      "[proc 0]sample: 61.509, forward: 321.581, backward: 48.550, update: 304.544\n",
      "[0]Valid average MRR: 0.47629350790171254\n",
      "[0]Valid average MR: 77.56693631857057\n",
      "[0]Valid average HITS@1: 0.3645748215939424\n",
      "[0]Valid average HITS@3: 0.5415985727515389\n",
      "[0]Valid average HITS@10: 0.6772416516860054\n",
      "[proc 0]validation take 439.936 seconds:\n",
      "[1]Valid average MRR: 0.477011472581617\n",
      "[1]Valid average MR: 75.71510721313932\n",
      "[1]Valid average HITS@1: 0.3652566782652512\n",
      "[1]Valid average HITS@3: 0.542017064218934\n",
      "[1]Valid average HITS@10: 0.677795406415765\n",
      "[proc 1]validation take 449.661 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4037976474329829\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4488480805769563\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.40404582445770504\n",
      "[proc 1][Train](60000/100000) average loss: 0.42632286406457426\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.44884360763132575\n",
      "[proc 1][Train](60000/100000) average regularization: 3.8185840741789435e-05\n",
      "[proc 1][Train] 20000 steps take 1208.069 seconds\n",
      "[proc 1]sample: 62.104, forward: 337.749, backward: 51.183, update: 307.105\n",
      "[proc 0][Train](60000/100000) average loss: 0.4264447161182761\n",
      "[proc 0][Train](60000/100000) average regularization: 3.8215851858331005e-05\n",
      "[proc 0][Train] 20000 steps take 1208.065 seconds\n",
      "[proc 0]sample: 61.849, forward: 324.751, backward: 48.807, update: 305.276\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.407033018925786\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.40726879912763836\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.450334083019197\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4504222878664732\n",
      "[proc 0][Train](80000/100000) average loss: 0.4288014410793781\n",
      "[proc 1][Train](80000/100000) average loss: 0.4287276533693075\n",
      "[proc 0][Train](80000/100000) average regularization: 3.996398504677927e-05\n",
      "[proc 1][Train](80000/100000) average regularization: 3.993174951265246e-05\n",
      "[proc 0][Train] 20000 steps take 762.514 seconds\n",
      "[proc 0]sample: 62.582, forward: 326.370, backward: 48.850, update: 306.421\n",
      "[proc 1][Train] 20000 steps take 762.520 seconds\n",
      "[proc 1]sample: 63.131, forward: 338.049, backward: 51.189, update: 309.664\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.4092066712528467\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4094640106067061\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4515770682543516\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4513223319336772\n",
      "[proc 1][Train](100000/100000) average loss: 0.4303918698310852\n",
      "[proc 0][Train](100000/100000) average loss: 0.4303931712716818\n",
      "[proc 1][Train](100000/100000) average regularization: 4.123768206318346e-05\n",
      "[proc 1][Train] 20000 steps take 764.411 seconds\n",
      "[proc 1]sample: 63.402, forward: 336.772, backward: 52.699, update: 311.256\n",
      "[proc 0][Train](100000/100000) average regularization: 4.1271620974112014e-05\n",
      "[proc 0][Train] 20000 steps take 764.414 seconds\n",
      "[proc 0]sample: 61.477, forward: 328.373, backward: 48.773, update: 308.087\n",
      "[0]Valid average MRR: 0.48122696036532836\n",
      "[0]Valid average MR: 76.20820055019884\n",
      "[0]Valid average HITS@1: 0.36836765266655774\n",
      "[0]Valid average HITS@3: 0.5478802364220733\n",
      "[0]Valid average HITS@10: 0.6833258430026693\n",
      "[proc 0]validation take 409.614 seconds:\n",
      "[1]Valid average MRR: 0.48289008172554343\n",
      "[1]Valid average MR: 74.24485383740645\n",
      "[1]Valid average HITS@1: 0.37130678142683016\n",
      "[1]Valid average HITS@3: 0.5483463505314694\n",
      "[1]Valid average HITS@10: 0.6831815984256794\n",
      "[proc 1]validation take 414.031 seconds:\n",
      "proc 1 takes 4663.267 seconds\n",
      "proc 0 takes 4663.276 seconds\n",
      "training takes 4663.798973798752 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_0\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.48047080018335264\n",
      "Test average MR : 75.62038751983222\n",
      "Test average HITS@1 : 0.3678169920398755\n",
      "Test average HITS@3 : 0.5470406585998625\n",
      "Test average HITS@10 : 0.6814911784933643\n",
      "-----------------------------------------\n",
      "testing takes 431.460 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 6.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: **6**, 12, 18\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.360 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.5036875696346164\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.5169191491127014\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.47066896205908854\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.47042986088337785\n",
      "[proc 0][Train](20000/100000) average loss: 0.48717826586812735\n",
      "[proc 0][Train](20000/100000) average regularization: 4.80664690672711e-05\n",
      "[proc 1][Train](20000/100000) average loss: 0.49367450483590364\n",
      "[proc 0][Train] 20000 steps take 764.606 seconds\n",
      "[proc 0]sample: 62.684, forward: 336.446, backward: 47.735, update: 315.841\n",
      "[proc 1][Train](20000/100000) average regularization: 6.0740869770393144e-05\n",
      "[proc 1][Train] 20000 steps take 764.595 seconds\n",
      "[proc 1]sample: 62.157, forward: 332.375, backward: 49.442, update: 315.076\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.43319438497424123\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.46474191436618567\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4333043588161469\n",
      "[proc 1][Train](40000/100000) average loss: 0.448968149676919\n",
      "[proc 1][Train](40000/100000) average regularization: 5.2248372380199727e-05\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4649584324374795\n",
      "[proc 1][Train] 20000 steps take 761.954 seconds\n",
      "[proc 1]sample: 62.645, forward: 332.269, backward: 49.768, update: 313.763\n",
      "[proc 0][Train](40000/100000) average loss: 0.44913139559030535\n",
      "[proc 0][Train](40000/100000) average regularization: 5.2266363208036635e-05\n",
      "[proc 0][Train] 20000 steps take 761.967 seconds\n",
      "[proc 0]sample: 62.957, forward: 334.026, backward: 47.698, update: 313.004\n",
      "[0]Valid average MRR: 0.4754670613662284\n",
      "[0]Valid average MR: 74.78554502369668\n",
      "[0]Valid average HITS@1: 0.3600295527591654\n",
      "[0]Valid average HITS@3: 0.5439886691725228\n",
      "[0]Valid average HITS@10: 0.6825563817617257\n",
      "[proc 0]validation take 367.199 seconds:\n",
      "[1]Valid average MRR: 0.47668476691273404\n",
      "[1]Valid average MR: 72.80435389528589\n",
      "[1]Valid average HITS@1: 0.36151834778049396\n",
      "[1]Valid average HITS@3: 0.5450097714102835\n",
      "[1]Valid average HITS@10: 0.6835493030635243\n",
      "[proc 1]validation take 382.752 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4322535188362002\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.4323852935254574\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.46369641656428573\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4638328105702996\n",
      "[proc 1][Train](60000/100000) average loss: 0.447974967764318\n",
      "[proc 0][Train](60000/100000) average loss: 0.4481090521395206\n",
      "[proc 0][Train](60000/100000) average regularization: 5.499461052622792e-05\n",
      "[proc 1][Train](60000/100000) average regularization: 5.4977971762491505e-05\n",
      "[proc 0][Train] 20000 steps take 1144.103 seconds\n",
      "[proc 0]sample: 61.801, forward: 332.377, backward: 47.819, update: 310.694\n",
      "[proc 1][Train] 20000 steps take 1144.113 seconds\n",
      "[proc 1]sample: 63.025, forward: 332.901, backward: 50.123, update: 313.170\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.43197506206333636\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.43197483823895455\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4631842679783702\n",
      "[proc 1][Train](80000/100000) average loss: 0.4475796650275588\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.46349201318323613\n",
      "[proc 1][Train](80000/100000) average regularization: 5.666297403822682e-05\n",
      "[proc 1][Train] 20000 steps take 757.792 seconds\n",
      "[proc 1]sample: 62.124, forward: 331.629, backward: 48.494, update: 314.432\n",
      "[proc 0][Train](80000/100000) average loss: 0.44773342572450636\n",
      "[proc 0][Train](80000/100000) average regularization: 5.6680070117727155e-05\n",
      "[proc 0][Train] 20000 steps take 757.806 seconds\n",
      "[proc 0]sample: 63.462, forward: 330.513, backward: 48.513, update: 309.736\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.43183663792163135\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.43152550989687444\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4630220052421093\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.46295954227298497\n",
      "[proc 1][Train](100000/100000) average loss: 0.44727375748604536\n",
      "[proc 0][Train](100000/100000) average loss: 0.447398090210557\n",
      "[proc 1][Train](100000/100000) average regularization: 5.786470170423854e-05\n",
      "[proc 0][Train](100000/100000) average regularization: 5.7887265606041186e-05\n",
      "[proc 1][Train] 20000 steps take 762.509 seconds\n",
      "[proc 1]sample: 62.402, forward: 332.106, backward: 48.887, update: 313.971\n",
      "[proc 0][Train] 20000 steps take 762.496 seconds\n",
      "[proc 0]sample: 63.189, forward: 334.295, backward: 49.969, update: 312.276\n",
      "[1]Valid average MRR: 0.48267444788162295\n",
      "[1]Valid average MR: 71.13128417440095\n",
      "[1]Valid average HITS@1: 0.36739140796829567\n",
      "[1]Valid average HITS@3: 0.5515569567674676\n",
      "[1]Valid average HITS@10: 0.6884520315681241\n",
      "[proc 1]validation take 374.549 seconds:\n",
      "[0]Valid average MRR: 0.48096923309855205\n",
      "[0]Valid average MR: 72.86914392329902\n",
      "[0]Valid average HITS@1: 0.36501062265075995\n",
      "[0]Valid average HITS@3: 0.551182791305769\n",
      "[0]Valid average HITS@10: 0.6870369613771313\n",
      "[proc 0]validation take 376.232 seconds:\n",
      "proc 0 takes 4567.211 seconds\n",
      "proc 1 takes 4567.197 seconds\n",
      "training takes 4567.69341635704 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_1\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.48156208624595787\n",
      "Test average MR : 72.63886638022021\n",
      "Test average HITS@1 : 0.36664067766602887\n",
      "Test average HITS@3 : 0.5505117222876676\n",
      "Test average HITS@10 : 0.6865454149274465\n",
      "-----------------------------------------\n",
      "testing takes 386.623 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 6.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: **6**, 12, 18\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.402 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.7042869151100516\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.7208506396114827\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4867796094735364\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4870771153839476\n",
      "[proc 1][Train](20000/100000) average loss: 0.5955332618713379\n",
      "[proc 1][Train](20000/100000) average regularization: 6.849490307904489e-05\n",
      "[proc 0][Train](20000/100000) average loss: 0.6039638776734472\n",
      "[proc 1][Train] 20000 steps take 767.264 seconds\n",
      "[proc 1]sample: 62.260, forward: 337.279, backward: 49.061, update: 318.117\n",
      "[proc 0][Train](20000/100000) average regularization: 7.918355824542687e-05\n",
      "[proc 0][Train] 20000 steps take 767.290 seconds\n",
      "[proc 0]sample: 63.769, forward: 330.973, backward: 52.355, update: 312.254\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4525718751937151\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.47578032491356137\n",
      "[proc 1][Train](40000/100000) average loss: 0.4641760999619961\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4520446507796645\n",
      "[proc 1][Train](40000/100000) average regularization: 5.739138630124217e-05\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4758840752974153\n",
      "[proc 1][Train] 20000 steps take 764.000 seconds\n",
      "[proc 1]sample: 62.647, forward: 335.540, backward: 48.803, update: 316.496\n",
      "[proc 0][Train](40000/100000) average loss: 0.46396436302810906\n",
      "[proc 0][Train](40000/100000) average regularization: 5.739394081647333e-05\n",
      "[proc 0][Train] 20000 steps take 764.007 seconds\n",
      "[proc 0]sample: 63.570, forward: 328.011, backward: 52.360, update: 309.662\n",
      "[0]Valid average MRR: 0.46358588799770123\n",
      "[0]Valid average MR: 76.18968921937136\n",
      "[0]Valid average HITS@1: 0.3492877376477638\n",
      "[0]Valid average HITS@3: 0.5298455630004902\n",
      "[0]Valid average HITS@10: 0.6705582339162173\n",
      "[proc 0]validation take 381.697 seconds:\n",
      "[1]Valid average MRR: 0.46511089084612284\n",
      "[1]Valid average MR: 74.5319426380765\n",
      "[1]Valid average HITS@1: 0.3509468394424508\n",
      "[1]Valid average HITS@3: 0.5313842717745835\n",
      "[1]Valid average HITS@10: 0.6720347004228603\n",
      "[proc 1]validation take 395.442 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4476668290480971\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4719582743048668\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.44679053225815296\n",
      "[proc 1][Train](60000/100000) average loss: 0.45981255165189505\n",
      "[proc 1][Train](60000/100000) average regularization: 6.029444417417835e-05\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.47285677358061073\n",
      "[proc 1][Train] 20000 steps take 1165.296 seconds\n",
      "[proc 1]sample: 62.835, forward: 337.668, backward: 49.853, update: 319.257\n",
      "[proc 0][Train](60000/100000) average loss: 0.4598236529618502\n",
      "[proc 0][Train](60000/100000) average regularization: 6.029837365495041e-05\n",
      "[proc 0][Train] 20000 steps take 1165.299 seconds\n",
      "[proc 0]sample: 62.719, forward: 325.175, backward: 51.638, update: 310.107\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.44520069753378627\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.47032444797754286\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.4442959581643343\n",
      "[proc 1][Train](80000/100000) average loss: 0.4577625727713108\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4712583421126008\n",
      "[proc 1][Train](80000/100000) average regularization: 6.199155297290417e-05\n",
      "[proc 1][Train] 20000 steps take 769.658 seconds\n",
      "[proc 1]sample: 62.816, forward: 336.290, backward: 49.335, update: 320.972\n",
      "[proc 0][Train](80000/100000) average loss: 0.45777715022265913\n",
      "[proc 0][Train](80000/100000) average regularization: 6.199189575927449e-05\n",
      "[proc 0][Train] 20000 steps take 769.654 seconds\n",
      "[proc 0]sample: 65.054, forward: 331.358, backward: 53.402, update: 309.103\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.4434478669285774\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.46966877851337197\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.442914758887887\n",
      "[proc 1][Train](100000/100000) average loss: 0.4565583226725459\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.46985172266215086\n",
      "[proc 1][Train](100000/100000) average regularization: 6.31471984419477e-05\n",
      "[proc 1][Train] 20000 steps take 766.511 seconds\n",
      "[proc 1]sample: 62.765, forward: 336.516, backward: 49.294, update: 317.691\n",
      "[proc 0][Train](100000/100000) average loss: 0.4563832408130169\n",
      "[proc 0][Train](100000/100000) average regularization: 6.315286577446386e-05\n",
      "[proc 0][Train] 20000 steps take 766.507 seconds\n",
      "[proc 0]sample: 62.309, forward: 324.012, backward: 51.307, update: 310.049\n",
      "[0]Valid average MRR: 0.47400738903192896\n",
      "[0]Valid average MR: 72.81550634635289\n",
      "[0]Valid average HITS@1: 0.359348613607888\n",
      "[0]Valid average HITS@3: 0.5415645257939751\n",
      "[0]Valid average HITS@10: 0.6806769897042\n",
      "[proc 0]validation take 378.855 seconds:\n",
      "[1]Valid average MRR: 0.4749349027381162\n",
      "[1]Valid average MR: 71.40871051430985\n",
      "[1]Valid average HITS@1: 0.3605173740441382\n",
      "[1]Valid average HITS@3: 0.5417719277937041\n",
      "[1]Valid average HITS@10: 0.6810230360146264\n",
      "[proc 1]validation take 387.301 seconds:\n",
      "proc 1 takes 4620.030 seconds\n",
      "proc 0 takes 4620.054 seconds\n",
      "training takes 4620.555933713913 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_2\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.4729662469408361\n",
      "Test average MR : 73.01831543610452\n",
      "Test average HITS@1 : 0.35811197287156893\n",
      "Test average HITS@3 : 0.5405888040747121\n",
      "Test average HITS@10 : 0.6793666628080377\n",
      "-----------------------------------------\n",
      "testing takes 397.855 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 6.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 6, **12**, 18\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.229 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.383959230658412\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.47511338955163956\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.38387153883874414\n",
      "[proc 1][Train](20000/100000) average loss: 0.4295363100335002\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4753732634086162\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00014209116112933772\n",
      "[proc 0][Train](20000/100000) average loss: 0.42962240105718374\n",
      "[proc 1][Train] 20000 steps take 768.691 seconds\n",
      "[proc 1]sample: 62.656, forward: 338.266, backward: 47.459, update: 320.065\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00014202867513868114\n",
      "[proc 0][Train] 20000 steps take 768.730 seconds\n",
      "[proc 0]sample: 62.849, forward: 328.302, backward: 50.074, update: 310.887\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.37546386020630595\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.46727114346921444\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3751138441979885\n",
      "[proc 1][Train](40000/100000) average loss: 0.42136750183403493\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4672573980867863\n",
      "[proc 1][Train](40000/100000) average regularization: 0.00013382493279641495\n",
      "[proc 1][Train] 20000 steps take 766.834 seconds\n",
      "[proc 1]sample: 63.417, forward: 337.785, backward: 48.777, update: 316.629\n",
      "[proc 0][Train](40000/100000) average loss: 0.4211856210261583\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00013381056471698684\n",
      "[proc 0][Train] 20000 steps take 766.836 seconds\n",
      "[proc 0]sample: 61.832, forward: 324.133, backward: 49.895, update: 309.409\n",
      "[1]Valid average MRR: 0.4899731714952621\n",
      "[1]Valid average MR: 72.79790544543332\n",
      "[1]Valid average HITS@1: 0.3750178745310064\n",
      "[1]Valid average HITS@3: 0.5584446093819158\n",
      "[1]Valid average HITS@10: 0.6959116691747754\n",
      "[proc 1]validation take 433.658 seconds:\n",
      "[0]Valid average MRR: 0.48915368543623\n",
      "[0]Valid average MR: 75.05628302554884\n",
      "[0]Valid average HITS@1: 0.3736449310889579\n",
      "[0]Valid average HITS@3: 0.558516505965027\n",
      "[0]Valid average HITS@10: 0.6960764286103394\n",
      "[proc 0]validation take 443.992 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3773867264926434\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.46718090292066333\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3770028724044561\n",
      "[proc 1][Train](60000/100000) average loss: 0.4222838147073984\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4672622829630971\n",
      "[proc 1][Train](60000/100000) average regularization: 0.00013681688951837714\n",
      "[proc 1][Train] 20000 steps take 1213.501 seconds\n",
      "[proc 1]sample: 62.455, forward: 337.324, backward: 48.885, update: 320.628\n",
      "[proc 0][Train](60000/100000) average loss: 0.422132577624917\n",
      "[proc 0][Train](60000/100000) average regularization: 0.00013680755263194443\n",
      "[proc 0][Train] 20000 steps take 1213.501 seconds\n",
      "[proc 0]sample: 62.954, forward: 329.109, backward: 50.702, update: 310.150\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3789978864222765\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4676327884733677\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3786080849990249\n",
      "[proc 1][Train](80000/100000) average loss: 0.42331533745527267\n",
      "[proc 1][Train](80000/100000) average regularization: 0.00013990351093016214\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4675508954539895\n",
      "[proc 1][Train] 20000 steps take 768.961 seconds\n",
      "[proc 1]sample: 62.932, forward: 339.080, backward: 48.490, update: 318.239\n",
      "[proc 0][Train](80000/100000) average loss: 0.4230794901430607\n",
      "[proc 0][Train](80000/100000) average regularization: 0.00013989546410084585\n",
      "[proc 0][Train] 20000 steps take 768.965 seconds\n",
      "[proc 0]sample: 61.698, forward: 324.453, backward: 50.001, update: 310.035\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.38014026039242743\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3797673763126135\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4678826333537698\n",
      "[proc 1][Train](100000/100000) average loss: 0.424011446814239\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.46779188132584093\n",
      "[proc 1][Train](100000/100000) average regularization: 0.00014252773121916108\n",
      "[proc 1][Train] 20000 steps take 765.524 seconds\n",
      "[proc 1]sample: 62.747, forward: 338.206, backward: 48.978, update: 315.365\n",
      "[proc 0][Train](100000/100000) average loss: 0.42377962881177667\n",
      "[proc 0][Train](100000/100000) average regularization: 0.00014252387555243332\n",
      "[proc 0][Train] 20000 steps take 765.525 seconds\n",
      "[proc 0]sample: 61.643, forward: 325.604, backward: 50.075, update: 309.780\n",
      "[1]Valid average MRR: 0.5014444223687231\n",
      "[1]Valid average MR: 70.2040590506411\n",
      "[1]Valid average HITS@1: 0.38596389685203975\n",
      "[1]Valid average HITS@3: 0.5716751669991897\n",
      "[1]Valid average HITS@10: 0.7074398905057301\n",
      "[proc 1]validation take 404.690 seconds:\n",
      "[0]Valid average MRR: 0.4999264299303186\n",
      "[0]Valid average MR: 72.35809228087378\n",
      "[0]Valid average HITS@1: 0.3838692324453887\n",
      "[0]Valid average HITS@3: 0.5706304134662527\n",
      "[0]Valid average HITS@10: 0.7062837064879882\n",
      "[proc 0]validation take 415.043 seconds:\n",
      "proc 0 takes 4698.600 seconds\n",
      "proc 1 takes 4698.566 seconds\n",
      "training takes 4699.09263253212 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_3\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.4996617058144997\n",
      "Test average MR : 71.56236508984931\n",
      "Test average HITS@1 : 0.38429901196401944\n",
      "Test average HITS@3 : 0.5700153891200284\n",
      "Test average HITS@10 : 0.705781814962855\n",
      "-----------------------------------------\n",
      "testing takes 417.976 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 12.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 6, **12**, 18\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.270 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.45208784756064413\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4973832110066549\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.4494747727662325\n",
      "[proc 1][Train](20000/100000) average loss: 0.4747355292841792\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.49731821014261807\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00017354594393436854\n",
      "[proc 1][Train] 20000 steps take 769.853 seconds\n",
      "[proc 1]sample: 62.285, forward: 339.908, backward: 47.746, update: 319.437\n",
      "[proc 0][Train](20000/100000) average loss: 0.47339649158865216\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00016649826855355058\n",
      "[proc 0][Train] 20000 steps take 769.904 seconds\n",
      "[proc 0]sample: 64.882, forward: 335.678, backward: 50.877, update: 314.269\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.39866700317263604\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3980675525352359\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4818078623071313\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4820775090664625\n",
      "[proc 1][Train](40000/100000) average loss: 0.44023743276894095\n",
      "[proc 1][Train](40000/100000) average regularization: 0.00017442756486998404\n",
      "[proc 0][Train](40000/100000) average loss: 0.4400725309118628\n",
      "[proc 1][Train] 20000 steps take 766.775 seconds\n",
      "[proc 1]sample: 63.490, forward: 337.786, backward: 48.931, update: 316.336\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00017447407655054123\n",
      "[proc 0][Train] 20000 steps take 766.777 seconds\n",
      "[proc 0]sample: 61.879, forward: 323.270, backward: 49.603, update: 309.257\n",
      "[1]Valid average MRR: 0.5074085104422458\n",
      "[1]Valid average MR: 65.5289295028497\n",
      "[1]Valid average HITS@1: 0.38961030117733575\n",
      "[1]Valid average HITS@3: 0.5796284821288736\n",
      "[1]Valid average HITS@10: 0.718202060507841\n",
      "[proc 1]validation take 377.258 seconds:\n",
      "[0]Valid average MRR: 0.5064762691118567\n",
      "[0]Valid average MR: 67.39318788473062\n",
      "[0]Valid average HITS@1: 0.38832938388625593\n",
      "[0]Valid average HITS@3: 0.5803474151549818\n",
      "[0]Valid average HITS@10: 0.716858691507327\n",
      "[proc 0]validation take 391.055 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.39835061751306056\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.39781162504255774\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.48030139823555945\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4797083692252636\n",
      "[proc 0][Train](60000/100000) average loss: 0.43905651165395976\n",
      "[proc 1][Train](60000/100000) average loss: 0.439029493303597\n",
      "[proc 0][Train](60000/100000) average regularization: 0.00018133720152909517\n",
      "[proc 1][Train](60000/100000) average regularization: 0.0001812925477112003\n",
      "[proc 0][Train] 20000 steps take 1158.120 seconds\n",
      "[proc 0]sample: 62.439, forward: 324.775, backward: 49.899, update: 307.905\n",
      "[proc 1][Train] 20000 steps take 1158.130 seconds\n",
      "[proc 1]sample: 63.383, forward: 338.143, backward: 49.311, update: 316.004\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.39830356698036196\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.39811436197310685\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.47915390323102475\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.47932922331988814\n",
      "[proc 0][Train](80000/100000) average loss: 0.4386341326609254\n",
      "[proc 1][Train](80000/100000) average loss: 0.43881639512330295\n",
      "[proc 0][Train](80000/100000) average regularization: 0.00018555370773319738\n",
      "[proc 0][Train] 20000 steps take 766.650 seconds\n",
      "[proc 0]sample: 62.884, forward: 329.969, backward: 49.995, update: 311.675\n",
      "[proc 1][Train](80000/100000) average regularization: 0.00018548178604469286\n",
      "[proc 1][Train] 20000 steps take 766.651 seconds\n",
      "[proc 1]sample: 64.013, forward: 336.661, backward: 49.905, update: 315.457\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3981801189944148\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.39831650520265105\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.47862678809911013\n",
      "[proc 0][Train](100000/100000) average loss: 0.438403453502059\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.47874884356409314\n",
      "[proc 0][Train](100000/100000) average regularization: 0.00018852892338982202\n",
      "[proc 1][Train](100000/100000) average loss: 0.4385326743617654\n",
      "[proc 0][Train] 20000 steps take 757.917 seconds\n",
      "[proc 0]sample: 63.015, forward: 331.861, backward: 49.100, update: 309.657\n",
      "[proc 1][Train](100000/100000) average regularization: 0.00018845737137962714\n",
      "[proc 1][Train] 20000 steps take 757.919 seconds\n",
      "[proc 1]sample: 62.311, forward: 331.712, backward: 50.012, update: 312.187\n",
      "[1]Valid average MRR: 0.5120793746352214\n",
      "[1]Valid average MR: 63.73809556234977\n",
      "[1]Valid average HITS@1: 0.393678884901639\n",
      "[1]Valid average HITS@3: 0.5855662310955555\n",
      "[1]Valid average HITS@10: 0.7233294974022348\n",
      "[proc 1]validation take 393.706 seconds:\n",
      "[0]Valid average MRR: 0.5121924790037987\n",
      "[0]Valid average MR: 65.47711703982132\n",
      "[0]Valid average HITS@1: 0.3939913929291278\n",
      "[0]Valid average HITS@3: 0.5857098109712916\n",
      "[0]Valid average HITS@10: 0.7226466742931852\n",
      "[proc 0]validation take 405.340 seconds:\n",
      "proc 0 takes 4624.709 seconds\n",
      "proc 1 takes 4624.665 seconds\n",
      "training takes 4625.166638612747 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_4\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5107944270862459\n",
      "Test average MR : 65.38785689480243\n",
      "Test average HITS@1 : 0.39288900086478684\n",
      "Test average HITS@3 : 0.5844307727925805\n",
      "Test average HITS@10 : 0.7206244850432734\n",
      "-----------------------------------------\n",
      "testing takes 390.006 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 12.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 6, **12**, 18\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.415 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.5818665719613433\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.5981306205287575\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5237261911897425\n",
      "[proc 1][Train](20000/100000) average loss: 0.5527963818743825\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00022862511255325444\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5237201938457147\n",
      "[proc 1][Train] 20000 steps take 776.585 seconds\n",
      "[proc 1]sample: 64.638, forward: 343.252, backward: 49.078, update: 319.393\n",
      "[proc 0][Train](20000/100000) average loss: 0.5609254071757197\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00023420683456670303\n",
      "[proc 0][Train] 20000 steps take 776.609 seconds\n",
      "[proc 0]sample: 64.021, forward: 331.983, backward: 50.714, update: 312.898\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.41681653038710353\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4165355801612139\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.49659958679825067\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4964855827212334\n",
      "[proc 1][Train](40000/100000) average loss: 0.45670805862993\n",
      "[proc 0][Train](40000/100000) average loss: 0.45651058143526313\n",
      "[proc 1][Train](40000/100000) average regularization: 0.00020338919439163875\n",
      "[proc 1][Train] 20000 steps take 770.739 seconds\n",
      "[proc 1]sample: 64.901, forward: 339.825, backward: 47.757, update: 318.024\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00020344115859188605\n",
      "[proc 0][Train] 20000 steps take 770.730 seconds\n",
      "[proc 0]sample: 62.550, forward: 327.066, backward: 50.512, update: 310.574\n",
      "[0]Valid average MRR: 0.49799636004176184\n",
      "[0]Valid average MR: 65.81666734760582\n",
      "[0]Valid average HITS@1: 0.37767949556027675\n",
      "[0]Valid average HITS@3: 0.5723531895189846\n",
      "[0]Valid average HITS@10: 0.7139544860271286\n",
      "[proc 0]validation take 436.442 seconds:\n",
      "[1]Valid average MRR: 0.4990277231155708\n",
      "[1]Valid average MR: 64.36792253688962\n",
      "[1]Valid average HITS@1: 0.37871194427231936\n",
      "[1]Valid average HITS@3: 0.5726454986823917\n",
      "[1]Valid average HITS@10: 0.7150289056701417\n",
      "[proc 1]validation take 439.179 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.412308617760241\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.412216503380239\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4913496365636587\n",
      "[proc 1][Train](60000/100000) average loss: 0.45182912707924844\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.49137504859268666\n",
      "[proc 1][Train](60000/100000) average regularization: 0.00020856084233600997\n",
      "[proc 0][Train](60000/100000) average loss: 0.45179577601403\n",
      "[proc 1][Train] 20000 steps take 1205.924 seconds\n",
      "[proc 1]sample: 63.108, forward: 337.677, backward: 47.197, update: 318.300\n",
      "[proc 0][Train](60000/100000) average regularization: 0.00020861522017221433\n",
      "[proc 0][Train] 20000 steps take 1205.925 seconds\n",
      "[proc 0]sample: 63.598, forward: 330.779, backward: 50.315, update: 312.488\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.40998270835578443\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.4102175404533744\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.48925267442911863\n",
      "[proc 1][Train](80000/100000) average loss: 0.44961769144833086\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.48837683709412816\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0002113836558186449\n",
      "[proc 1][Train] 20000 steps take 763.211 seconds\n",
      "[proc 1]sample: 62.803, forward: 335.303, backward: 47.034, update: 316.488\n",
      "[proc 0][Train](80000/100000) average loss: 0.4492971887901425\n",
      "[proc 0][Train](80000/100000) average regularization: 0.0002114547817436687\n",
      "[proc 0][Train] 20000 steps take 763.219 seconds\n",
      "[proc 0]sample: 64.110, forward: 331.695, backward: 50.472, update: 312.518\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.4085247073084116\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4875357592448592\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4088708845153451\n",
      "[proc 1][Train](100000/100000) average loss: 0.4480302332788706\n",
      "[proc 1][Train](100000/100000) average regularization: 0.00021322558543470223\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.48667657118290664\n",
      "[proc 1][Train] 20000 steps take 768.339 seconds\n",
      "[proc 1]sample: 62.747, forward: 338.306, backward: 46.746, update: 317.143\n",
      "[proc 0][Train](100000/100000) average loss: 0.44777372785508635\n",
      "[proc 0][Train](100000/100000) average regularization: 0.00021330986631874112\n",
      "[proc 0][Train] 20000 steps take 768.338 seconds\n",
      "[proc 0]sample: 64.980, forward: 338.296, backward: 50.618, update: 313.515\n",
      "[0]Valid average MRR: 0.50933856403605\n",
      "[0]Valid average MR: 63.334324099798444\n",
      "[0]Valid average HITS@1: 0.3905730783897151\n",
      "[0]Valid average HITS@3: 0.5840278912676363\n",
      "[0]Valid average HITS@10: 0.721141798768862\n",
      "[proc 0]validation take 432.483 seconds:\n",
      "[1]Valid average MRR: 0.5097445143374414\n",
      "[1]Valid average MR: 62.208682595994745\n",
      "[1]Valid average HITS@1: 0.3909415281532375\n",
      "[1]Valid average HITS@3: 0.5834349060650837\n",
      "[1]Valid average HITS@10: 0.7216680171867872\n",
      "[proc 1]validation take 446.092 seconds:\n",
      "proc 1 takes 4730.890 seconds\n",
      "proc 0 takes 4730.901 seconds\n",
      "training takes 4731.375898599625 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_5\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5081618799691081\n",
      "Test average MR : 63.346902088426155\n",
      "Test average HITS@1 : 0.3892698339200719\n",
      "Test average HITS@3 : 0.5824049926118605\n",
      "Test average HITS@10 : 0.7203010411488727\n",
      "-----------------------------------------\n",
      "testing takes 449.777 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 12.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 6, 12, **18**\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.312 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.3974419813364744\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3973693295776844\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.49413016603253784\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4943271064866334\n",
      "[proc 0][Train](20000/100000) average loss: 0.4457860736370087\n",
      "[proc 1][Train](20000/100000) average loss: 0.44584821795225144\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00034383239237577073\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00034392451061903556\n",
      "[proc 0][Train] 20000 steps take 773.587 seconds\n",
      "[proc 0]sample: 63.842, forward: 337.941, backward: 48.612, update: 321.045\n",
      "[proc 1][Train] 20000 steps take 773.554 seconds\n",
      "[proc 1]sample: 63.118, forward: 336.176, backward: 51.348, update: 314.914\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3793200508564711\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.37934330805838107\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4806205401942134\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4805397278368473\n",
      "[proc 0][Train](40000/100000) average loss: 0.4299819240361452\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00034111375186621445\n",
      "[proc 1][Train](40000/100000) average loss: 0.42992988927960396\n",
      "[proc 0][Train] 20000 steps take 761.568 seconds\n",
      "[proc 0]sample: 63.416, forward: 333.057, backward: 48.375, update: 311.008\n",
      "[proc 1][Train](40000/100000) average regularization: 0.0003411513713464956\n",
      "[proc 1][Train] 20000 steps take 761.573 seconds\n",
      "[proc 1]sample: 62.760, forward: 334.095, backward: 51.082, update: 312.920\n",
      "[0]Valid average MRR: 0.47517700254711237\n",
      "[0]Valid average MR: 79.56937408073215\n",
      "[0]Valid average HITS@1: 0.3605640899929182\n",
      "[0]Valid average HITS@3: 0.5415713351854878\n",
      "[0]Valid average HITS@10: 0.6828866372500954\n",
      "[proc 0]validation take 398.668 seconds:\n",
      "[1]Valid average MRR: 0.47662789555522445\n",
      "[1]Valid average MR: 78.09107839599066\n",
      "[1]Valid average HITS@1: 0.36160005992223726\n",
      "[1]Valid average HITS@3: 0.5441075331785342\n",
      "[1]Valid average HITS@10: 0.684805627242828\n",
      "[proc 1]validation take 417.662 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.37871345772445203\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4795833254665136\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.378664480535686\n",
      "[proc 0][Train](60000/100000) average loss: 0.4291483915567398\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.47954806516468523\n",
      "[proc 0][Train](60000/100000) average regularization: 0.00034414298355550275\n",
      "[proc 0][Train] 20000 steps take 1176.956 seconds\n",
      "[proc 0]sample: 62.238, forward: 333.450, backward: 47.952, update: 309.815\n",
      "[proc 1][Train](60000/100000) average loss: 0.4291062728717923\n",
      "[proc 1][Train](60000/100000) average regularization: 0.00034417743578815137\n",
      "[proc 1][Train] 20000 steps take 1176.954 seconds\n",
      "[proc 1]sample: 62.145, forward: 333.713, backward: 51.102, update: 311.497\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.37886430414915084\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4794901545166969\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.378916520871222\n",
      "[proc 1][Train](80000/100000) average loss: 0.4291772293180227\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4793244331330061\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0003484197703117388\n",
      "[proc 1][Train] 20000 steps take 761.854 seconds\n",
      "[proc 1]sample: 63.449, forward: 334.035, backward: 51.716, update: 311.279\n",
      "[proc 0][Train](80000/100000) average loss: 0.4291204769551754\n",
      "[proc 0][Train](80000/100000) average regularization: 0.00034839188632759035\n",
      "[proc 0][Train] 20000 steps take 761.867 seconds\n",
      "[proc 0]sample: 63.073, forward: 333.196, backward: 48.087, update: 309.710\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3790986198797822\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4793138354241848\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.37907201729416845\n",
      "[proc 0][Train](100000/100000) average loss: 0.4292062276139855\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4794553680494428\n",
      "[proc 0][Train](100000/100000) average regularization: 0.0003524487492060871\n",
      "[proc 1][Train](100000/100000) average loss: 0.42926369264274833\n",
      "[proc 0][Train] 20000 steps take 755.573 seconds\n",
      "[proc 0]sample: 62.476, forward: 332.498, backward: 48.145, update: 307.429\n",
      "[proc 1][Train](100000/100000) average regularization: 0.0003524714787476114\n",
      "[proc 1][Train] 20000 steps take 755.584 seconds\n",
      "[proc 1]sample: 62.043, forward: 331.001, backward: 51.489, update: 309.080\n",
      "[1]Valid average MRR: 0.49119904221329336\n",
      "[1]Valid average MR: 74.14914508671701\n",
      "[1]Valid average HITS@1: 0.375303867027108\n",
      "[1]Valid average HITS@3: 0.5607836194393185\n",
      "[1]Valid average HITS@10: 0.6986251932151685\n",
      "[proc 1]validation take 413.544 seconds:\n",
      "[0]Valid average MRR: 0.4905121523890639\n",
      "[0]Valid average MR: 75.50917905975922\n",
      "[0]Valid average HITS@1: 0.3750749033066405\n",
      "[0]Valid average HITS@3: 0.5598545513972871\n",
      "[0]Valid average HITS@10: 0.6968390804597702\n",
      "[proc 0]validation take 415.517 seconds:\n",
      "proc 0 takes 4645.068 seconds\n",
      "proc 1 takes 4645.032 seconds\n",
      "training takes 4645.6271443367 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_6\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.49017364555859255\n",
      "Test average MR : 75.4888752323689\n",
      "Test average HITS@1 : 0.3751489544250529\n",
      "Test average HITS@3 : 0.5589008355066494\n",
      "Test average HITS@10 : 0.6963151228746332\n",
      "-----------------------------------------\n",
      "testing takes 406.526 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 18.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 6, 12, **18**\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.383 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.44481785182058814\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.44526457321047785\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5225160657379544\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5223184525237419\n",
      "[proc 1][Train](20000/100000) average loss: 0.48366695875674487\n",
      "[proc 0][Train](20000/100000) average loss: 0.48379151276051996\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00045024607891391497\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00044773521695096863\n",
      "[proc 1][Train] 20000 steps take 768.523 seconds\n",
      "[proc 1]sample: 64.674, forward: 336.847, backward: 51.486, update: 314.929\n",
      "[proc 0][Train] 20000 steps take 768.548 seconds\n",
      "[proc 0]sample: 63.776, forward: 336.221, backward: 50.086, update: 310.458\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3934228162139654\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3933695258826017\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.496493106277287\n",
      "[proc 1][Train](40000/100000) average loss: 0.44495796127021314\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.49661079469174146\n",
      "[proc 1][Train](40000/100000) average regularization: 0.000427544902243244\n",
      "[proc 1][Train] 20000 steps take 762.428 seconds\n",
      "[proc 1]sample: 64.210, forward: 333.533, backward: 51.271, update: 312.771\n",
      "[proc 0][Train](40000/100000) average loss: 0.44499016021490095\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00042754195692104986\n",
      "[proc 0][Train] 20000 steps take 762.443 seconds\n",
      "[proc 0]sample: 63.708, forward: 333.563, backward: 49.049, update: 309.130\n",
      "[0]Valid average MRR: 0.5033679972445089\n",
      "[0]Valid average MR: 68.65744675055836\n",
      "[0]Valid average HITS@1: 0.3846523124693577\n",
      "[0]Valid average HITS@3: 0.576445633818162\n",
      "[0]Valid average HITS@10: 0.7160109222639864\n",
      "[proc 0]validation take 431.941 seconds:\n",
      "[1]Valid average MRR: 0.504774403904549\n",
      "[1]Valid average MR: 67.0493745616484\n",
      "[1]Valid average HITS@1: 0.38594687348917656\n",
      "[1]Valid average HITS@3: 0.5782121383386559\n",
      "[1]Valid average HITS@10: 0.7176334801882103\n",
      "[proc 1]validation take 437.286 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3917388369038701\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4928920511379838\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.39170263621360063\n",
      "[proc 1][Train](60000/100000) average loss: 0.4423154440894723\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4930963625997305\n",
      "[proc 1][Train](60000/100000) average regularization: 0.0004377544935079641\n",
      "[proc 1][Train] 20000 steps take 1201.802 seconds\n",
      "[proc 1]sample: 62.908, forward: 335.088, backward: 51.443, update: 314.403\n",
      "[proc 0][Train](60000/100000) average loss: 0.44239949943423273\n",
      "[proc 0][Train](60000/100000) average regularization: 0.00043774366863799516\n",
      "[proc 0][Train] 20000 steps take 1201.793 seconds\n",
      "[proc 0]sample: 63.913, forward: 331.816, backward: 49.697, update: 310.669\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3911573596715927\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3911154298812151\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.49138433164805173\n",
      "[proc 1][Train](80000/100000) average loss: 0.4412708455950022\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4913850101187825\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0004449620725310524\n",
      "[proc 0][Train](80000/100000) average loss: 0.4412502199873328\n",
      "[proc 1][Train] 20000 steps take 755.346 seconds\n",
      "[proc 1]sample: 61.885, forward: 332.757, backward: 48.460, update: 312.004\n",
      "[proc 0][Train](80000/100000) average regularization: 0.0004449610354975448\n",
      "[proc 0][Train] 20000 steps take 755.343 seconds\n",
      "[proc 0]sample: 63.278, forward: 325.575, backward: 51.115, update: 306.188\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.39073256964236497\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4901646845117211\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.39069911620765924\n",
      "[proc 1][Train](100000/100000) average loss: 0.4404486270964146\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4903534659147263\n",
      "[proc 1][Train](100000/100000) average regularization: 0.0004503882392004016\n",
      "[proc 0][Train](100000/100000) average loss: 0.4405262911200523\n",
      "[proc 1][Train] 20000 steps take 758.755 seconds\n",
      "[proc 1]sample: 61.822, forward: 334.561, backward: 48.297, update: 313.832\n",
      "[proc 0][Train](100000/100000) average regularization: 0.00045039482798747487\n",
      "[proc 0][Train] 20000 steps take 758.754 seconds\n",
      "[proc 0]sample: 63.147, forward: 326.716, backward: 50.551, update: 308.023\n",
      "[1]Valid average MRR: 0.5130088873645207\n",
      "[1]Valid average MR: 64.62505021892045\n",
      "[1]Valid average HITS@1: 0.39454367173508925\n",
      "[1]Valid average HITS@3: 0.587248139346439\n",
      "[1]Valid average HITS@10: 0.7241908795631125\n",
      "[proc 1]validation take 441.840 seconds:\n",
      "[0]Valid average MRR: 0.5118707047974405\n",
      "[0]Valid average MR: 66.35279457427684\n",
      "[0]Valid average HITS@1: 0.3934466416081059\n",
      "[0]Valid average HITS@3: 0.5861830636814295\n",
      "[0]Valid average HITS@10: 0.7227692433404151\n",
      "[proc 0]validation take 446.658 seconds:\n",
      "proc 0 takes 4693.540 seconds\n",
      "proc 1 takes 4693.516 seconds\n",
      "training takes 4694.041881084442 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_7\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5105659365012417\n",
      "Test average MR : 65.84928876389958\n",
      "Test average HITS@1 : 0.39187951544699945\n",
      "Test average HITS@3 : 0.5849431760147626\n",
      "Test average HITS@10 : 0.7224476872059215\n",
      "-----------------------------------------\n",
      "testing takes 431.517 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 18.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 6, 12, **18**\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.296 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.5509488394364714\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5502117491080658\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.5708548694066703\n",
      "[proc 1][Train](20000/100000) average loss: 0.5505802944555879\n",
      "[proc 1][Train](20000/100000) average regularization: 0.0004945536896291742\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5501031103353016\n",
      "[proc 1][Train] 20000 steps take 767.935 seconds\n",
      "[proc 1]sample: 62.055, forward: 339.232, backward: 48.865, update: 317.563\n",
      "[proc 0][Train](20000/100000) average loss: 0.5604789897054434\n",
      "[proc 0][Train](20000/100000) average regularization: 0.0005566750022273482\n",
      "[proc 0][Train] 20000 steps take 767.974 seconds\n",
      "[proc 0]sample: 63.950, forward: 337.041, backward: 49.892, update: 310.222\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4087001942396164\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4087513010978699\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5102731790944934\n",
      "[proc 1][Train](40000/100000) average loss: 0.4594866867482662\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5106419847816229\n",
      "[proc 1][Train](40000/100000) average regularization: 0.0004858422531557153\n",
      "[proc 1][Train] 20000 steps take 762.636 seconds\n",
      "[proc 1]sample: 62.115, forward: 337.220, backward: 48.061, update: 315.014\n",
      "[proc 0][Train](40000/100000) average loss: 0.45969664301127194\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00048587471369828565\n",
      "[proc 0][Train] 20000 steps take 762.632 seconds\n",
      "[proc 0]sample: 64.420, forward: 334.175, backward: 51.387, update: 307.386\n",
      "[0]Valid average MRR: 0.4987278974708764\n",
      "[0]Valid average MR: 66.25837214686496\n",
      "[0]Valid average HITS@1: 0.37778504112872474\n",
      "[0]Valid average HITS@3: 0.5731635071090048\n",
      "[0]Valid average HITS@10: 0.7170050934248515\n",
      "[proc 0]validation take 421.025 seconds:\n",
      "[1]Valid average MRR: 0.4996727050026955\n",
      "[1]Valid average MR: 64.59680505525783\n",
      "[1]Valid average HITS@1: 0.37873237230775514\n",
      "[1]Valid average HITS@3: 0.5742490994641045\n",
      "[1]Valid average HITS@10: 0.7180624689323628\n",
      "[proc 1]validation take 428.651 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4034951689898968\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5042736698523164\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.4038974793046713\n",
      "[proc 1][Train](60000/100000) average loss: 0.45388441933989526\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5036882311388851\n",
      "[proc 1][Train](60000/100000) average regularization: 0.0004953958804937428\n",
      "[proc 1][Train] 20000 steps take 1194.391 seconds\n",
      "[proc 1]sample: 62.631, forward: 336.546, backward: 49.517, update: 316.042\n",
      "[proc 0][Train](60000/100000) average loss: 0.4537928552344441\n",
      "[proc 0][Train](60000/100000) average regularization: 0.0004954586396823288\n",
      "[proc 0][Train] 20000 steps take 1194.389 seconds\n",
      "[proc 0]sample: 62.776, forward: 334.337, backward: 49.914, update: 314.086\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.40138944568485024\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.40137129058092835\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.5009527512654662\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.5009775071009993\n",
      "[proc 0][Train](80000/100000) average loss: 0.45117109861969945\n",
      "[proc 0][Train](80000/100000) average regularization: 0.0005003796658740612\n",
      "[proc 1][Train](80000/100000) average loss: 0.45117439877390864\n",
      "[proc 0][Train] 20000 steps take 757.352 seconds\n",
      "[proc 0]sample: 62.941, forward: 332.659, backward: 48.591, update: 308.997\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0005003177121718182\n",
      "[proc 1][Train] 20000 steps take 757.364 seconds\n",
      "[proc 1]sample: 62.474, forward: 331.481, backward: 49.785, update: 311.980\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.39963528998047115\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4985002615764737\n",
      "[proc 1][Train](100000/100000) average loss: 0.44906777578741314\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3995605353027582\n",
      "[proc 1][Train](100000/100000) average regularization: 0.0005033878185291542\n",
      "[proc 1][Train] 20000 steps take 754.578 seconds\n",
      "[proc 1]sample: 61.404, forward: 330.455, backward: 49.271, update: 312.375\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4987461027547717\n",
      "[proc 0][Train](100000/100000) average loss: 0.4491533190712333\n",
      "[proc 0][Train](100000/100000) average regularization: 0.0005034282440727111\n",
      "[proc 0][Train] 20000 steps take 754.593 seconds\n",
      "[proc 0]sample: 62.586, forward: 329.898, backward: 49.331, update: 309.705\n",
      "[1]Valid average MRR: 0.509035085275767\n",
      "[1]Valid average MR: 61.944459576322544\n",
      "[1]Valid average HITS@1: 0.3891540750526022\n",
      "[1]Valid average HITS@3: 0.5840000817121418\n",
      "[1]Valid average HITS@10: 0.7242487589968473\n",
      "[proc 1]validation take 415.436 seconds:\n",
      "[0]Valid average MRR: 0.507942494897664\n",
      "[0]Valid average MR: 63.83500163425396\n",
      "[0]Valid average HITS@1: 0.38797189083183525\n",
      "[0]Valid average HITS@3: 0.5830745764558479\n",
      "[0]Valid average HITS@10: 0.7229190499536962\n",
      "[proc 0]validation take 421.308 seconds:\n",
      "proc 0 takes 4658.249 seconds\n",
      "proc 1 takes 4658.224 seconds\n",
      "training takes 4658.771943569183 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_8\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5080507676831657\n",
      "Test average MR : 63.48034822991073\n",
      "Test average HITS@1 : 0.3882058737411223\n",
      "Test average HITS@3 : 0.5832936121533192\n",
      "Test average HITS@10 : 0.7227421913834546\n",
      "-----------------------------------------\n",
      "testing takes 427.850 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 18.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: **6**, 12, 18\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.393 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.4181524638861418\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.4171834736675024\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.44899489559736105\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4488053493257612\n",
      "[proc 0][Train](20000/100000) average loss: 0.4335736798301339\n",
      "[proc 0][Train](20000/100000) average regularization: 8.71257054541843e-06\n",
      "[proc 1][Train](20000/100000) average loss: 0.4329944115921855\n",
      "[proc 0][Train] 20000 steps take 1072.781 seconds\n",
      "[proc 0]sample: 65.626, forward: 409.670, backward: 52.087, update: 524.366\n",
      "[proc 1][Train](20000/100000) average regularization: 8.632654604690514e-06\n",
      "[proc 1][Train] 20000 steps take 1072.783 seconds\n",
      "[proc 1]sample: 65.842, forward: 427.366, backward: 51.122, update: 528.015\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.41158650361299515\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.41144523613899947\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.44981242423802614\n",
      "[proc 1][Train](40000/100000) average loss: 0.4306994638696313\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4497117220312357\n",
      "[proc 0][Train](40000/100000) average loss: 0.4305784792140126\n",
      "[proc 1][Train](40000/100000) average regularization: 1.0343609531400944e-05\n",
      "[proc 1][Train] 20000 steps take 1065.862 seconds\n",
      "[proc 1]sample: 65.097, forward: 426.214, backward: 51.696, update: 522.223\n",
      "[proc 0][Train](40000/100000) average regularization: 1.0344360621320447e-05\n",
      "[proc 0][Train] 20000 steps take 1065.876 seconds\n",
      "[proc 0]sample: 65.270, forward: 407.645, backward: 52.078, update: 521.527\n",
      "[0]Valid average MRR: 0.49049752309526096\n",
      "[0]Valid average MR: 73.59966906357248\n",
      "[0]Valid average HITS@1: 0.378350220624285\n",
      "[0]Valid average HITS@3: 0.5576721414174429\n",
      "[0]Valid average HITS@10: 0.6897709320695102\n",
      "[proc 0]validation take 413.384 seconds:\n",
      "[1]Valid average MRR: 0.49255734564778125\n",
      "[1]Valid average MR: 71.75057368732848\n",
      "[1]Valid average HITS@1: 0.38056749082440744\n",
      "[1]Valid average HITS@3: 0.5595306999325875\n",
      "[1]Valid average HITS@10: 0.6914889995029178\n",
      "[proc 1]validation take 422.280 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.41466631939709186\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.41459778072088954\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.45113562806546686\n",
      "[proc 1][Train](60000/100000) average loss: 0.43290097377598286\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4510752943634987\n",
      "[proc 1][Train](60000/100000) average regularization: 1.1036829275326454e-05\n",
      "[proc 0][Train](60000/100000) average loss: 0.432836537578702\n",
      "[proc 1][Train] 20000 steps take 1484.389 seconds\n",
      "[proc 1]sample: 65.215, forward: 424.546, backward: 52.607, update: 519.449\n",
      "[proc 0][Train](60000/100000) average regularization: 1.103803892419819e-05\n",
      "[proc 0][Train] 20000 steps take 1484.391 seconds\n",
      "[proc 0]sample: 64.324, forward: 420.524, backward: 50.174, update: 516.761\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.41656240944117306\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.4165092080786824\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4518729590058327\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4520574099138379\n",
      "[proc 1][Train](80000/100000) average loss: 0.4343099096536636\n",
      "[proc 0][Train](80000/100000) average loss: 0.43419108368605375\n",
      "[proc 1][Train](80000/100000) average regularization: 1.1480629402421983e-05\n",
      "[proc 0][Train](80000/100000) average regularization: 1.1482363743607493e-05\n",
      "[proc 1][Train] 20000 steps take 1059.716 seconds\n",
      "[proc 1]sample: 65.350, forward: 421.397, backward: 53.435, update: 519.238\n",
      "[proc 0][Train] 20000 steps take 1059.711 seconds\n",
      "[proc 0]sample: 63.889, forward: 420.289, backward: 49.489, update: 516.475\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.41770618929713965\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4526386621147394\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4176480584204197\n",
      "[proc 1][Train](100000/100000) average loss: 0.43517242572158576\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.45231831545084716\n",
      "[proc 1][Train](100000/100000) average regularization: 1.1808189784824208e-05\n",
      "[proc 1][Train] 20000 steps take 1079.466 seconds\n",
      "[proc 1]sample: 66.304, forward: 433.038, backward: 54.423, update: 525.396\n",
      "[proc 0][Train](100000/100000) average loss: 0.43498318695425986\n",
      "[proc 0][Train](100000/100000) average regularization: 1.1809556219168372e-05\n",
      "[proc 0][Train] 20000 steps take 1079.473 seconds\n",
      "[proc 0]sample: 65.552, forward: 423.652, backward: 50.412, update: 519.463\n",
      "[0]Valid average MRR: 0.4952289118431002\n",
      "[0]Valid average MR: 72.78330473388898\n",
      "[0]Valid average HITS@1: 0.38342662199705835\n",
      "[0]Valid average HITS@3: 0.562438715476385\n",
      "[0]Valid average HITS@10: 0.693189246608923\n",
      "[proc 0]validation take 418.858 seconds:\n",
      "[1]Valid average MRR: 0.4960628596106997\n",
      "[1]Valid average MR: 70.92829078627508\n",
      "[1]Valid average HITS@1: 0.3839551400341829\n",
      "[1]Valid average HITS@3: 0.5629455865229441\n",
      "[1]Valid average HITS@10: 0.6950162402881714\n",
      "[proc 1]validation take 422.185 seconds:\n",
      "proc 1 takes 6184.402 seconds\n",
      "proc 0 takes 6184.410 seconds\n",
      "training takes 6184.948779344559 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_9\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.49530409384180774\n",
      "Test average MR : 72.42085668371273\n",
      "Test average HITS@1 : 0.3837338363169614\n",
      "Test average HITS@3 : 0.5621795351941004\n",
      "Test average HITS@10 : 0.693211763824673\n",
      "-----------------------------------------\n",
      "testing takes 411.827 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 6.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 11\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: **6**, 12, 18\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.480 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.7217826963886619\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.7166496900141239\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4821169642751198\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.48304385173818554\n",
      "[proc 1][Train](20000/100000) average loss: 0.6019498302489519\n",
      "[proc 1][Train](20000/100000) average regularization: 4.433065932680052e-05\n",
      "[proc 0][Train](20000/100000) average loss: 0.5998467705622316\n",
      "[proc 1][Train] 20000 steps take 1075.631 seconds\n",
      "[proc 1]sample: 66.519, forward: 421.792, backward: 54.216, update: 532.846\n",
      "[proc 0][Train](20000/100000) average regularization: 2.673227263398985e-05\n",
      "[proc 0][Train] 20000 steps take 1075.673 seconds\n",
      "[proc 0]sample: 64.437, forward: 420.677, backward: 51.910, update: 516.132\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4507472341492772\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4503597481161356\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.471499454203248\n",
      "[proc 1][Train](40000/100000) average loss: 0.46112334422171114\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.47241401953250167\n",
      "[proc 1][Train](40000/100000) average regularization: 1.4601171597360007e-05\n",
      "[proc 1][Train] 20000 steps take 1068.325 seconds\n",
      "[proc 1]sample: 66.509, forward: 417.612, backward: 51.699, update: 531.588\n",
      "[proc 0][Train](40000/100000) average loss: 0.46138688383996485\n",
      "[proc 0][Train](40000/100000) average regularization: 1.459551999064388e-05\n",
      "[proc 0][Train] 20000 steps take 1068.338 seconds\n",
      "[proc 0]sample: 63.987, forward: 417.714, backward: 49.086, update: 515.546\n",
      "[1]Valid average MRR: 0.4707362303005299\n",
      "[1]Valid average MR: 74.04717173849392\n",
      "[1]Valid average HITS@1: 0.3561355604431522\n",
      "[1]Valid average HITS@3: 0.5376692973436744\n",
      "[1]Valid average HITS@10: 0.6768591214582893\n",
      "[proc 1]validation take 463.800 seconds:\n",
      "[0]Valid average MRR: 0.4682773490952511\n",
      "[0]Valid average MR: 75.79579315792341\n",
      "[0]Valid average HITS@1: 0.35320654246336547\n",
      "[0]Valid average HITS@3: 0.5352147682083129\n",
      "[0]Valid average HITS@10: 0.6766390205371248\n",
      "[proc 0]validation take 463.925 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.44560397359132764\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.44558518288731574\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.46891214436888695\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.46866149607598784\n",
      "[proc 1][Train](60000/100000) average loss: 0.4572486635476351\n",
      "[proc 0][Train](60000/100000) average loss: 0.4571327348202467\n",
      "[proc 0][Train](60000/100000) average regularization: 1.5420262568795805e-05\n",
      "[proc 1][Train](60000/100000) average regularization: 1.5423818027602465e-05\n",
      "[proc 0][Train] 20000 steps take 1524.285 seconds\n",
      "[proc 0]sample: 64.561, forward: 416.730, backward: 50.020, update: 519.808\n",
      "[proc 1][Train] 20000 steps take 1524.305 seconds\n",
      "[proc 1]sample: 65.780, forward: 414.766, backward: 53.178, update: 520.786\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.4430045769661665\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.443275909397006\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4673407512560487\n",
      "[proc 0][Train](80000/100000) average loss: 0.45517266407758\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4670895785540342\n",
      "[proc 0][Train](80000/100000) average regularization: 1.5939119621634746e-05\n",
      "[proc 1][Train](80000/100000) average loss: 0.4551827438592911\n",
      "[proc 0][Train] 20000 steps take 1064.166 seconds\n",
      "[proc 0]sample: 66.385, forward: 419.647, backward: 51.863, update: 526.011\n",
      "[proc 1][Train](80000/100000) average regularization: 1.5943392497774767e-05\n",
      "[proc 1][Train] 20000 steps take 1064.175 seconds\n",
      "[proc 1]sample: 65.088, forward: 408.619, backward: 51.990, update: 518.378\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.44153155110776426\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.44162460782527924\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.46620951845943925\n",
      "[proc 0][Train](100000/100000) average loss: 0.45387053480893375\n",
      "[proc 0][Train](100000/100000) average regularization: 1.6316633435508264e-05\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4661641523078084\n",
      "[proc 0][Train] 20000 steps take 1064.348 seconds\n",
      "[proc 0]sample: 65.608, forward: 419.326, backward: 53.336, update: 523.964\n",
      "[proc 1][Train](100000/100000) average loss: 0.4538943800970912\n",
      "[proc 1][Train](100000/100000) average regularization: 1.631997820632023e-05\n",
      "[proc 1][Train] 20000 steps take 1064.346 seconds\n",
      "[proc 1]sample: 65.707, forward: 412.954, backward: 51.697, update: 520.525\n",
      "[1]Valid average MRR: 0.4857781925826427\n",
      "[1]Valid average MR: 71.27453577289472\n",
      "[1]Valid average HITS@1: 0.3735334372893359\n",
      "[1]Valid average HITS@3: 0.5521391557773889\n",
      "[1]Valid average HITS@10: 0.6859189551740809\n",
      "[proc 1]validation take 454.320 seconds:\n",
      "[0]Valid average MRR: 0.484052056738569\n",
      "[0]Valid average MR: 73.14981001797679\n",
      "[0]Valid average HITS@1: 0.37175532494416297\n",
      "[0]Valid average HITS@3: 0.5506414446805034\n",
      "[0]Valid average HITS@10: 0.6845379146919431\n",
      "[proc 0]validation take 459.137 seconds:\n",
      "proc 0 takes 6255.947 seconds\n",
      "proc 1 takes 6255.912 seconds\n",
      "training takes 6256.456567525864 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_10\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.4843010456223453\n",
      "Test average MR : 72.96058751029913\n",
      "Test average HITS@1 : 0.37218348461428463\n",
      "Test average HITS@3 : 0.5507347283411754\n",
      "Test average HITS@10 : 0.6841366090823046\n",
      "-----------------------------------------\n",
      "testing takes 475.170 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 6.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: **6**, 12, 18\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.406 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 1.4184481607005\n",
      "[proc 0][Train](20000/100000) average pos_loss: 1.4677627029448748\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.49592609119911196\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4955773651834118\n",
      "[proc 1][Train](20000/100000) average loss: 0.9571871260970831\n",
      "[proc 1][Train](20000/100000) average regularization: 6.130790451701386e-05\n",
      "[proc 0][Train](20000/100000) average loss: 0.9816700342416763\n",
      "[proc 1][Train] 20000 steps take 1064.919 seconds\n",
      "[proc 1]sample: 65.263, forward: 423.968, backward: 50.471, update: 524.908\n",
      "[proc 0][Train](20000/100000) average regularization: 6.272481984813112e-05\n",
      "[proc 0][Train] 20000 steps take 1064.935 seconds\n",
      "[proc 0]sample: 64.834, forward: 409.127, backward: 48.326, update: 525.806\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4818683433234692\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4825094736918807\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4881757433876395\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.48818741454929115\n",
      "[proc 1][Train](40000/100000) average loss: 0.4850220433548093\n",
      "[proc 0][Train](40000/100000) average loss: 0.48534844419658185\n",
      "[proc 1][Train](40000/100000) average regularization: 1.6230784499884974e-05\n",
      "[proc 0][Train](40000/100000) average regularization: 1.623094847473112e-05\n",
      "[proc 1][Train] 20000 steps take 1055.243 seconds\n",
      "[proc 1]sample: 65.043, forward: 415.342, backward: 51.188, update: 523.379\n",
      "[proc 0][Train] 20000 steps take 1055.239 seconds\n",
      "[proc 0]sample: 63.864, forward: 400.524, backward: 47.799, update: 521.213\n",
      "[1]Valid average MRR: 0.43339858565372047\n",
      "[1]Valid average MR: 80.33475762135956\n",
      "[1]Valid average HITS@1: 0.3181598425679402\n",
      "[1]Valid average HITS@3: 0.4964727592147463\n",
      "[1]Valid average HITS@10: 0.6468094813321803\n",
      "[proc 1]validation take 435.734 seconds:\n",
      "[0]Valid average MRR: 0.43181112455392867\n",
      "[0]Valid average MR: 82.5428106444408\n",
      "[0]Valid average HITS@1: 0.31644604238165275\n",
      "[0]Valid average HITS@3: 0.49482145775453507\n",
      "[0]Valid average HITS@10: 0.6457005501988342\n",
      "[proc 0]validation take 446.509 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.47008208245188\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.46906662123054266\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.48219123685359955\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.48082827161848546\n",
      "[proc 1][Train](60000/100000) average loss: 0.4756289289727807\n",
      "[proc 0][Train](60000/100000) average loss: 0.4754551769480109\n",
      "[proc 1][Train](60000/100000) average regularization: 1.7371940907287352e-05\n",
      "[proc 0][Train](60000/100000) average regularization: 1.7374018413829618e-05\n",
      "[proc 1][Train] 20000 steps take 1504.269 seconds\n",
      "[proc 1]sample: 64.635, forward: 410.083, backward: 51.968, update: 525.252\n",
      "[proc 0][Train] 20000 steps take 1504.270 seconds\n",
      "[proc 0]sample: 63.199, forward: 405.593, backward: 49.772, update: 527.208\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.46373434480726716\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4776064847767353\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.46305922529101373\n",
      "[proc 0][Train](80000/100000) average loss: 0.470670414917171\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4788107758447528\n",
      "[proc 1][Train](80000/100000) average loss: 0.4709350005030632\n",
      "[proc 0][Train](80000/100000) average regularization: 1.8084605732929047e-05\n",
      "[proc 1][Train](80000/100000) average regularization: 1.808092430637771e-05\n",
      "[proc 0][Train] 20000 steps take 1060.784 seconds\n",
      "[proc 0]sample: 63.243, forward: 414.599, backward: 51.457, update: 531.223\n",
      "[proc 1][Train] 20000 steps take 1060.787 seconds\n",
      "[proc 1]sample: 63.700, forward: 405.068, backward: 48.343, update: 528.318\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4598327654004097\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.4590977603465319\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.476498923073709\n",
      "[proc 1][Train](100000/100000) average loss: 0.46779834172725676\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4755315105304122\n",
      "[proc 1][Train](100000/100000) average regularization: 1.8580488403949857e-05\n",
      "[proc 1][Train] 20000 steps take 1064.781 seconds\n",
      "[proc 1]sample: 64.215, forward: 406.954, backward: 48.471, update: 529.560\n",
      "[proc 0][Train](100000/100000) average loss: 0.4676821380108595\n",
      "[proc 0][Train](100000/100000) average regularization: 1.8583204398055388e-05\n",
      "[proc 0][Train] 20000 steps take 1064.788 seconds\n",
      "[proc 0]sample: 63.736, forward: 415.677, backward: 51.522, update: 533.580\n",
      "[1]Valid average MRR: 0.4531364398013237\n",
      "[1]Valid average MR: 75.21384407961486\n",
      "[1]Valid average HITS@1: 0.3366199772567872\n",
      "[1]Valid average HITS@3: 0.5197641242841676\n",
      "[1]Valid average HITS@10: 0.6654534683399498\n",
      "[proc 1]validation take 427.564 seconds:\n",
      "[0]Valid average MRR: 0.451064186312205\n",
      "[0]Valid average MR: 77.35509274391241\n",
      "[0]Valid average HITS@1: 0.3349097074685406\n",
      "[0]Valid average HITS@3: 0.5172209511358065\n",
      "[0]Valid average HITS@10: 0.6634900855259575\n",
      "[proc 0]validation take 435.373 seconds:\n",
      "proc 0 takes 6185.389 seconds\n",
      "proc 1 takes 6185.379 seconds\n",
      "training takes 6185.92497420311 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_11\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.4519306662904462\n",
      "Test average MR : 77.1317642332337\n",
      "Test average HITS@1 : 0.33600713619371225\n",
      "Test average HITS@3 : 0.5181860585467496\n",
      "Test average HITS@10 : 0.6632676685483158\n",
      "-----------------------------------------\n",
      "testing takes 465.952 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 6.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 6, **12**, 18\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.262 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.38020947179198267\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.37905955603569746\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4643195164442062\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4645797241859138\n",
      "[proc 0][Train](20000/100000) average loss: 0.4222644941106439\n",
      "[proc 1][Train](20000/100000) average loss: 0.4218196400269866\n",
      "[proc 0][Train](20000/100000) average regularization: 3.46332848541806e-05\n",
      "[proc 1][Train](20000/100000) average regularization: 3.4648022888609374e-05\n",
      "[proc 1][Train] 20000 steps take 1070.700 seconds\n",
      "[proc 1]sample: 65.279, forward: 417.271, backward: 52.265, update: 529.262\n",
      "[proc 0][Train] 20000 steps take 1070.705 seconds\n",
      "[proc 0]sample: 64.293, forward: 424.324, backward: 52.366, update: 528.420\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.370741416798532\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3710994465187192\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.45711103543937204\n",
      "[proc 1][Train](40000/100000) average loss: 0.41392622617930175\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4571658642798662\n",
      "[proc 1][Train](40000/100000) average regularization: 3.559821567087056e-05\n",
      "[proc 0][Train](40000/100000) average loss: 0.4141326555565\n",
      "[proc 1][Train] 20000 steps take 1055.709 seconds\n",
      "[proc 1]sample: 64.398, forward: 407.191, backward: 51.132, update: 524.349\n",
      "[proc 0][Train](40000/100000) average regularization: 3.557267807846074e-05\n",
      "[proc 0][Train] 20000 steps take 1055.714 seconds\n",
      "[proc 0]sample: 64.377, forward: 413.918, backward: 53.276, update: 522.412\n",
      "[1]Valid average MRR: 0.525148817858829\n",
      "[1]Valid average MR: 65.26181932083591\n",
      "[1]Valid average HITS@1: 0.4100213132503047\n",
      "[1]Valid average HITS@3: 0.5969582655236046\n",
      "[1]Valid average HITS@10: 0.7277351437112293\n",
      "[proc 1]validation take 472.643 seconds:\n",
      "[0]Valid average MRR: 0.5245258130308291\n",
      "[0]Valid average MR: 66.83607751811299\n",
      "[0]Valid average HITS@1: 0.40978237184725175\n",
      "[0]Valid average HITS@3: 0.5966150514789998\n",
      "[0]Valid average HITS@10: 0.7262386283161737\n",
      "[proc 0]validation take 485.593 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.37385835647433996\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.37352049713581803\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4577174573376775\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4576852501749992\n",
      "[proc 0][Train](60000/100000) average loss: 0.4157879069492221\n",
      "[proc 0][Train](60000/100000) average regularization: 3.7153164553819804e-05\n",
      "[proc 1][Train](60000/100000) average loss: 0.415602873775363\n",
      "[proc 0][Train] 20000 steps take 1544.004 seconds\n",
      "[proc 0]sample: 63.779, forward: 414.185, backward: 52.819, update: 522.236\n",
      "[proc 1][Train](60000/100000) average regularization: 3.7181242046062835e-05\n",
      "[proc 1][Train] 20000 steps take 1544.018 seconds\n",
      "[proc 1]sample: 64.612, forward: 412.291, backward: 51.166, update: 527.784\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.37533459306657313\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4581468212470412\n",
      "[proc 1][Train](80000/100000) average loss: 0.4167407070964575\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.37570225551873443\n",
      "[proc 1][Train](80000/100000) average regularization: 3.83086230371191e-05\n",
      "[proc 1][Train] 20000 steps take 1044.203 seconds\n",
      "[proc 1]sample: 63.114, forward: 398.280, backward: 49.231, update: 527.230\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.45832926215976477\n",
      "[proc 0][Train](80000/100000) average loss: 0.41701575883179903\n",
      "[proc 0][Train](80000/100000) average regularization: 3.827968283439987e-05\n",
      "[proc 0][Train] 20000 steps take 1044.230 seconds\n",
      "[proc 0]sample: 64.169, forward: 405.535, backward: 54.094, update: 518.602\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.37668654420524833\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4587925212264061\n",
      "[proc 1][Train](100000/100000) average loss: 0.4177395327448845\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.37702918346375225\n",
      "[proc 1][Train](100000/100000) average regularization: 3.9178518204789724e-05\n",
      "[proc 1][Train] 20000 steps take 1049.324 seconds\n",
      "[proc 1]sample: 63.712, forward: 402.026, backward: 49.459, update: 529.482\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.45860520503520963\n",
      "[proc 0][Train](100000/100000) average loss: 0.41781719430685044\n",
      "[proc 0][Train](100000/100000) average regularization: 3.915003394358791e-05\n",
      "[proc 0][Train] 20000 steps take 1049.321 seconds\n",
      "[proc 0]sample: 64.485, forward: 409.078, backward: 53.727, update: 519.376\n",
      "[1]Valid average MRR: 0.5330231677081421\n",
      "[1]Valid average MR: 63.89646050239349\n",
      "[1]Valid average HITS@1: 0.41905050491294255\n",
      "[1]Valid average HITS@3: 0.6050920282996384\n",
      "[1]Valid average HITS@10: 0.7329681254553749\n",
      "[proc 1]validation take 471.832 seconds:\n",
      "[0]Valid average MRR: 0.5316331356014069\n",
      "[0]Valid average MR: 65.63173108351037\n",
      "[0]Valid average HITS@1: 0.4170241597210873\n",
      "[0]Valid average HITS@3: 0.6045445878956256\n",
      "[0]Valid average HITS@10: 0.7324079370267473\n",
      "[proc 0]validation take 479.051 seconds:\n",
      "proc 0 takes 6243.025 seconds\n",
      "proc 1 takes 6243.021 seconds\n",
      "training takes 6243.5432851314545 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_12\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5317226639143606\n",
      "Test average MR : 65.37202686967595\n",
      "Test average HITS@1 : 0.41791504660996753\n",
      "Test average HITS@3 : 0.6036978148811429\n",
      "Test average HITS@10 : 0.7314768788685592\n",
      "-----------------------------------------\n",
      "testing takes 465.245 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 12.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 6, **12**, 18\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.474 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.5985839904025197\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5075471606007893\n",
      "[proc 1][Train](20000/100000) average loss: 0.5530655757650733\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.6014358468756079\n",
      "[proc 1][Train](20000/100000) average regularization: 5.8779624448357024e-05\n",
      "[proc 1][Train] 20000 steps take 1079.414 seconds\n",
      "[proc 1]sample: 65.721, forward: 422.491, backward: 53.744, update: 537.164\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5098049956214891\n",
      "[proc 0][Train](20000/100000) average loss: 0.5556204212322832\n",
      "[proc 0][Train](20000/100000) average regularization: 6.380058353420282e-05\n",
      "[proc 0][Train] 20000 steps take 1079.434 seconds\n",
      "[proc 0]sample: 65.864, forward: 414.163, backward: 50.705, update: 524.194\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4080887965336442\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.40774016328454016\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.48341188240498306\n",
      "[proc 1][Train](40000/100000) average loss: 0.4457503393903375\n",
      "[proc 1][Train](40000/100000) average regularization: 4.925860592920799e-05\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4848298275887966\n",
      "[proc 1][Train] 20000 steps take 1078.852 seconds\n",
      "[proc 1]sample: 65.696, forward: 423.101, backward: 50.694, update: 539.057\n",
      "[proc 0][Train](40000/100000) average loss: 0.44628499549627304\n",
      "[proc 0][Train](40000/100000) average regularization: 4.9240368044956995e-05\n",
      "[proc 0][Train] 20000 steps take 1078.852 seconds\n",
      "[proc 0]sample: 64.611, forward: 406.564, backward: 49.265, update: 519.754\n",
      "[1]Valid average MRR: 0.516224155930039\n",
      "[1]Valid average MR: 64.02284875763497\n",
      "[1]Valid average HITS@1: 0.39723676774004646\n",
      "[1]Valid average HITS@3: 0.5908741156362992\n",
      "[1]Valid average HITS@10: 0.7285658838189532\n",
      "[proc 1]validation take 431.498 seconds:\n",
      "[0]Valid average MRR: 0.5155131492488039\n",
      "[0]Valid average MR: 65.41000027237565\n",
      "[0]Valid average HITS@1: 0.39676281527482704\n",
      "[0]Valid average HITS@3: 0.5904423380726699\n",
      "[0]Valid average HITS@10: 0.726453124148826\n",
      "[proc 0]validation take 436.763 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.40398567146509884\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4035520130202174\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4794166973203421\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.47939718369692563\n",
      "[proc 0][Train](60000/100000) average loss: 0.4417011843904853\n",
      "[proc 1][Train](60000/100000) average loss: 0.4414745983496308\n",
      "[proc 0][Train](60000/100000) average regularization: 5.0937474677084535e-05\n",
      "[proc 0][Train] 20000 steps take 1498.597 seconds\n",
      "[proc 0]sample: 64.086, forward: 405.773, backward: 49.369, update: 516.400\n",
      "[proc 1][Train](60000/100000) average regularization: 5.0945401040371506e-05\n",
      "[proc 1][Train] 20000 steps take 1498.621 seconds\n",
      "[proc 1]sample: 64.424, forward: 422.923, backward: 51.495, update: 522.713\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.40172529213130476\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4770433586746454\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.40206689463704826\n",
      "[proc 1][Train](80000/100000) average loss: 0.4393843254059553\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4772325573414564\n",
      "[proc 1][Train](80000/100000) average regularization: 5.198936955257523e-05\n",
      "[proc 1][Train] 20000 steps take 1043.245 seconds\n",
      "[proc 1]sample: 62.614, forward: 417.432, backward: 51.179, update: 511.750\n",
      "[proc 0][Train](80000/100000) average loss: 0.4396497259438038\n",
      "[proc 0][Train](80000/100000) average regularization: 5.19815667246803e-05\n",
      "[proc 0][Train] 20000 steps take 1043.262 seconds\n",
      "[proc 0]sample: 63.732, forward: 403.333, backward: 49.254, update: 511.859\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.4003563660115004\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.40077863395810126\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4755936736106873\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.47551412011832\n",
      "[proc 1][Train](100000/100000) average loss: 0.43797501974850894\n",
      "[proc 0][Train](100000/100000) average loss: 0.4381463769555092\n",
      "[proc 1][Train](100000/100000) average regularization: 5.2745911589227034e-05\n",
      "[proc 0][Train](100000/100000) average regularization: 5.274166597828298e-05\n",
      "[proc 1][Train] 20000 steps take 1053.311 seconds\n",
      "[proc 1]sample: 63.661, forward: 419.441, backward: 51.882, update: 518.031\n",
      "[proc 0][Train] 20000 steps take 1053.304 seconds\n",
      "[proc 0]sample: 63.957, forward: 404.525, backward: 49.566, update: 515.160\n",
      "[0]Valid average MRR: 0.5255667998225652\n",
      "[0]Valid average MR: 62.922151631530205\n",
      "[0]Valid average HITS@1: 0.4068611428882715\n",
      "[0]Valid average HITS@3: 0.6016403824154274\n",
      "[0]Valid average HITS@10: 0.7349716729313068\n",
      "[proc 0]validation take 405.157 seconds:\n",
      "[1]Valid average MRR: 0.5262901176734012\n",
      "[1]Valid average MR: 61.68584405237748\n",
      "[1]Valid average HITS@1: 0.4081930040787977\n",
      "[1]Valid average HITS@3: 0.6013502931423085\n",
      "[1]Valid average HITS@10: 0.7356407934248963\n",
      "[proc 1]validation take 413.396 seconds:\n",
      "proc 1 takes 6166.840 seconds\n",
      "proc 0 takes 6166.844 seconds\n",
      "training takes 6167.369866609573 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_13\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5245343016405659\n",
      "Test average MR : 62.50214153904819\n",
      "Test average HITS@1 : 0.4060157159685953\n",
      "Test average HITS@3 : 0.6005314693885889\n",
      "Test average HITS@10 : 0.7332558202877629\n",
      "-----------------------------------------\n",
      "testing takes 412.014 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 12.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 15\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 6, **12**, 18\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.396 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 1.0328068170055746\n",
      "[proc 0][Train](20000/100000) average pos_loss: 1.0720008361011744\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.543161665487136\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5472656722327285\n",
      "[proc 1][Train](20000/100000) average loss: 0.7879842409074307\n",
      "[proc 0][Train](20000/100000) average loss: 0.8096332546219229\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00011665164409560021\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00013777154079273258\n",
      "[proc 1][Train] 20000 steps take 1078.273 seconds\n",
      "[proc 1]sample: 65.787, forward: 422.176, backward: 49.862, update: 538.500\n",
      "[proc 0][Train] 20000 steps take 1078.276 seconds\n",
      "[proc 0]sample: 65.366, forward: 428.496, backward: 50.679, update: 526.753\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.43609850381165743\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4347990807771683\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5048878045409918\n",
      "[proc 1][Train](40000/100000) average loss: 0.47049315413832665\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5074595971509814\n",
      "[proc 1][Train](40000/100000) average regularization: 5.817737538145593e-05\n",
      "[proc 0][Train](40000/100000) average loss: 0.4711293390259147\n",
      "[proc 1][Train] 20000 steps take 1078.742 seconds\n",
      "[proc 1]sample: 66.878, forward: 422.306, backward: 51.603, update: 537.444\n",
      "[proc 0][Train](40000/100000) average regularization: 5.8169443765291364e-05\n",
      "[proc 0][Train] 20000 steps take 1078.745 seconds\n",
      "[proc 0]sample: 65.053, forward: 426.755, backward: 49.572, update: 525.860\n",
      "[1]Valid average MRR: 0.4932635636868638\n",
      "[1]Valid average MR: 65.20520982997066\n",
      "[1]Valid average HITS@1: 0.37296826164227787\n",
      "[1]Valid average HITS@3: 0.5658225348468238\n",
      "[1]Valid average HITS@10: 0.7106436873965831\n",
      "[proc 1]validation take 436.666 seconds:\n",
      "[0]Valid average MRR: 0.4917587046847014\n",
      "[0]Valid average MR: 66.03954213651468\n",
      "[0]Valid average HITS@1: 0.3719357738192515\n",
      "[0]Valid average HITS@3: 0.5643249169254235\n",
      "[0]Valid average HITS@10: 0.7093955984093261\n",
      "[proc 0]validation take 437.494 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.42451965854763984\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.42406501163691285\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.49605543164759874\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.49663812493383885\n",
      "[proc 0][Train](60000/100000) average loss: 0.46035156835317614\n",
      "[proc 1][Train](60000/100000) average loss: 0.46028754515200854\n",
      "[proc 0][Train](60000/100000) average regularization: 5.964434712113871e-05\n",
      "[proc 1][Train](60000/100000) average regularization: 5.964314383254532e-05\n",
      "[proc 0][Train] 20000 steps take 1512.313 seconds\n",
      "[proc 0]sample: 65.514, forward: 423.595, backward: 50.460, update: 528.229\n",
      "[proc 1][Train] 20000 steps take 1512.320 seconds\n",
      "[proc 1]sample: 65.420, forward: 421.816, backward: 50.145, update: 536.349\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.41883165269345046\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.41907574070096015\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.49123080793619156\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4917152877166867\n",
      "[proc 1][Train](80000/100000) average loss: 0.4551532743185759\n",
      "[proc 0][Train](80000/100000) average loss: 0.45527347020953896\n",
      "[proc 1][Train](80000/100000) average regularization: 6.048767546108138e-05\n",
      "[proc 0][Train](80000/100000) average regularization: 6.049177903532836e-05\n",
      "[proc 1][Train] 20000 steps take 1069.976 seconds\n",
      "[proc 1]sample: 65.530, forward: 422.013, backward: 50.992, update: 529.639\n",
      "[proc 0][Train] 20000 steps take 1069.977 seconds\n",
      "[proc 0]sample: 66.691, forward: 421.212, backward: 51.778, update: 528.139\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4158068261548877\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.415341542544961\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4883894109264016\n",
      "[proc 0][Train](100000/100000) average loss: 0.4520981185719371\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4885046529337764\n",
      "[proc 0][Train](100000/100000) average regularization: 6.104841737269453e-05\n",
      "[proc 1][Train](100000/100000) average loss: 0.45192309770435096\n",
      "[proc 0][Train] 20000 steps take 1079.749 seconds\n",
      "[proc 0]sample: 64.237, forward: 428.452, backward: 49.062, update: 524.032\n",
      "[proc 1][Train](100000/100000) average regularization: 6.103764616127591e-05\n",
      "[proc 1][Train] 20000 steps take 1079.754 seconds\n",
      "[proc 1]sample: 66.474, forward: 430.655, backward: 51.462, update: 530.802\n",
      "[0]Valid average MRR: 0.5037199338747503\n",
      "[0]Valid average MR: 62.75795336928692\n",
      "[0]Valid average HITS@1: 0.381656180203737\n",
      "[0]Valid average HITS@3: 0.5797549980933704\n",
      "[0]Valid average HITS@10: 0.722231301410906\n",
      "[proc 0]validation take 429.115 seconds:\n",
      "[1]Valid average MRR: 0.5054789886326427\n",
      "[1]Valid average MR: 61.4538224258973\n",
      "[1]Valid average HITS@1: 0.38422070449484874\n",
      "[1]Valid average HITS@3: 0.5806158371749389\n",
      "[1]Valid average HITS@10: 0.7240002178990447\n",
      "[proc 1]validation take 437.292 seconds:\n",
      "proc 1 takes 6256.357 seconds\n",
      "proc 0 takes 6256.357 seconds\n",
      "training takes 6256.897694349289 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_14\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5037679764020483\n",
      "Test average MR : 62.41803931715887\n",
      "Test average HITS@1 : 0.38211321217238536\n",
      "Test average HITS@3 : 0.5794054760753659\n",
      "Test average HITS@10 : 0.7218263344614149\n",
      "-----------------------------------------\n",
      "testing takes 441.597 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 12.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 16\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 6, 12, **18**\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.454 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3829809942625463\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.48243878317261113\n",
      "[proc 1][Train](20000/100000) average loss: 0.4327098887875676\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.38390373805090783\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00010234642183477262\n",
      "[proc 1][Train] 20000 steps take 1067.177 seconds\n",
      "[proc 1]sample: 65.626, forward: 424.215, backward: 51.735, update: 525.317\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4825698373047635\n",
      "[proc 0][Train](20000/100000) average loss: 0.43323678770363333\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00010239179209174835\n",
      "[proc 0][Train] 20000 steps take 1067.205 seconds\n",
      "[proc 0]sample: 65.564, forward: 413.802, backward: 50.128, update: 523.742\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.36506072981655596\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3649875722378492\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4674492164850235\n",
      "[proc 1][Train](40000/100000) average loss: 0.41625497331023215\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.46751130569577215\n",
      "[proc 1][Train](40000/100000) average regularization: 8.757159871056502e-05\n",
      "[proc 0][Train](40000/100000) average loss: 0.4162494389399886\n",
      "[proc 0][Train](40000/100000) average regularization: 8.756787410966354e-05\n",
      "[proc 1][Train] 20000 steps take 1065.010 seconds\n",
      "[proc 1]sample: 64.264, forward: 421.449, backward: 50.602, update: 528.106\n",
      "[proc 0][Train] 20000 steps take 1064.995 seconds\n",
      "[proc 0]sample: 66.057, forward: 413.000, backward: 49.966, update: 523.045\n",
      "[1]Valid average MRR: 0.5171017884600975\n",
      "[1]Valid average MR: 67.41907774229352\n",
      "[1]Valid average HITS@1: 0.3998345329129698\n",
      "[1]Valid average HITS@3: 0.5895701260409787\n",
      "[1]Valid average HITS@10: 0.7259749279911751\n",
      "[proc 1]validation take 474.121 seconds:\n",
      "[0]Valid average MRR: 0.5153697443533701\n",
      "[0]Valid average MR: 69.06548932287411\n",
      "[0]Valid average HITS@1: 0.39774336765266655\n",
      "[0]Valid average HITS@3: 0.5885459225363622\n",
      "[0]Valid average HITS@10: 0.7249550580160157\n",
      "[proc 0]validation take 474.462 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3651182865306735\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.36501015226691963\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4659006202325225\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4661597977608442\n",
      "[proc 1][Train](60000/100000) average loss: 0.41550945327877997\n",
      "[proc 0][Train](60000/100000) average loss: 0.4155849749237299\n",
      "[proc 1][Train](60000/100000) average regularization: 8.84066009599337e-05\n",
      "[proc 0][Train](60000/100000) average regularization: 8.839986713755934e-05\n",
      "[proc 1][Train] 20000 steps take 1544.656 seconds\n",
      "[proc 1]sample: 64.542, forward: 422.811, backward: 50.464, update: 529.869\n",
      "[proc 0][Train] 20000 steps take 1544.653 seconds\n",
      "[proc 0]sample: 64.445, forward: 423.960, backward: 52.205, update: 527.016\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.365617931894958\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.465591236127913\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3655215806439519\n",
      "[proc 1][Train](80000/100000) average loss: 0.4156045839935541\n",
      "[proc 1][Train](80000/100000) average regularization: 8.987560125278832e-05\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.46551412797421216\n",
      "[proc 1][Train] 20000 steps take 1070.472 seconds\n",
      "[proc 1]sample: 64.270, forward: 422.495, backward: 50.194, update: 528.994\n",
      "[proc 0][Train](80000/100000) average loss: 0.415517854359746\n",
      "[proc 0][Train](80000/100000) average regularization: 8.987382863197126e-05\n",
      "[proc 0][Train] 20000 steps take 1070.487 seconds\n",
      "[proc 0]sample: 65.819, forward: 421.432, backward: 52.444, update: 527.835\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.3659714435786009\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.46523533328473565\n",
      "[proc 1][Train](100000/100000) average loss: 0.4156033885464072\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.36592974985688925\n",
      "[proc 1][Train](100000/100000) average regularization: 9.117419589274505e-05\n",
      "[proc 1][Train] 20000 steps take 1072.229 seconds\n",
      "[proc 1]sample: 63.998, forward: 426.364, backward: 49.978, update: 529.695\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.46530368357598784\n",
      "[proc 0][Train](100000/100000) average loss: 0.415616716773808\n",
      "[proc 0][Train](100000/100000) average regularization: 9.116618211010064e-05\n",
      "[proc 0][Train] 20000 steps take 1072.224 seconds\n",
      "[proc 0]sample: 66.536, forward: 415.169, backward: 52.727, update: 530.181\n",
      "[1]Valid average MRR: 0.5283970961559467\n",
      "[1]Valid average MR: 65.72206976855036\n",
      "[1]Valid average HITS@1: 0.4122275410773746\n",
      "[1]Valid average HITS@3: 0.6008702343095664\n",
      "[1]Valid average HITS@10: 0.7349019794766337\n",
      "[proc 1]validation take 455.781 seconds:\n",
      "[0]Valid average MRR: 0.5268956976889748\n",
      "[0]Valid average MR: 67.38687217410252\n",
      "[0]Valid average HITS@1: 0.4102113635125565\n",
      "[0]Valid average HITS@3: 0.6000980552377839\n",
      "[0]Valid average HITS@10: 0.734181783515825\n",
      "[proc 0]validation take 455.872 seconds:\n",
      "proc 0 takes 6275.437 seconds\n",
      "proc 1 takes 6275.426 seconds\n",
      "training takes 6275.9313859939575 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_15\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5260299740859465\n",
      "Test average MR : 66.92618840096148\n",
      "Test average HITS@1 : 0.4095446590901353\n",
      "Test average HITS@3 : 0.5991440653152387\n",
      "Test average HITS@10 : 0.7324301871888981\n",
      "-----------------------------------------\n",
      "testing takes 474.638 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 18.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 17\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 6, 12, **18**\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.460 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.5428887829340995\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.5252985682189465\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.526477529769842\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5265404814787035\n",
      "[proc 0][Train](20000/100000) average loss: 0.5346831559851766\n",
      "[proc 1][Train](20000/100000) average loss: 0.5259195253863931\n",
      "[proc 0][Train](20000/100000) average regularization: 0.00013194052623266542\n",
      "[proc 1][Train](20000/100000) average regularization: 0.0001291605656795582\n",
      "[proc 0][Train] 20000 steps take 1066.221 seconds\n",
      "[proc 0]sample: 65.263, forward: 422.888, backward: 50.737, update: 525.919\n",
      "[proc 1][Train] 20000 steps take 1066.222 seconds\n",
      "[proc 1]sample: 65.086, forward: 413.627, backward: 52.241, update: 521.350\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.392028975597024\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3919323775485158\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.49101434246897696\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.49124325442463157\n",
      "[proc 1][Train](40000/100000) average loss: 0.4415216591283679\n",
      "[proc 0][Train](40000/100000) average loss: 0.4415878160133958\n",
      "[proc 1][Train](40000/100000) average regularization: 0.00011545010573972832\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00011546754755727307\n",
      "[proc 1][Train] 20000 steps take 1058.790 seconds\n",
      "[proc 1]sample: 64.674, forward: 407.647, backward: 51.716, update: 518.029\n",
      "[proc 0][Train] 20000 steps take 1058.793 seconds\n",
      "[proc 0]sample: 65.198, forward: 415.915, backward: 51.658, update: 525.193\n",
      "[0]Valid average MRR: 0.5197825841747589\n",
      "[0]Valid average MR: 65.5789174429373\n",
      "[0]Valid average HITS@1: 0.3988805360352999\n",
      "[0]Valid average HITS@3: 0.5967274064389606\n",
      "[0]Valid average HITS@10: 0.735448330337201\n",
      "[proc 0]validation take 429.214 seconds:\n",
      "[1]Valid average MRR: 0.5207740884095432\n",
      "[1]Valid average MR: 64.14652348883608\n",
      "[1]Valid average HITS@1: 0.40046439733890793\n",
      "[1]Valid average HITS@3: 0.5963181870799485\n",
      "[1]Valid average HITS@10: 0.7363421559748599\n",
      "[proc 1]validation take 434.327 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.387461909173429\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3876289525702596\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.48506244428008793\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.48529654521644117\n",
      "[proc 0][Train](60000/100000) average loss: 0.4364627489641309\n",
      "[proc 0][Train](60000/100000) average regularization: 0.00011883871680256562\n",
      "[proc 1][Train](60000/100000) average loss: 0.43626217666268347\n",
      "[proc 0][Train] 20000 steps take 1488.553 seconds\n",
      "[proc 0]sample: 63.619, forward: 419.520, backward: 49.885, update: 517.627\n",
      "[proc 1][Train](60000/100000) average regularization: 0.00011881844245835964\n",
      "[proc 1][Train] 20000 steps take 1488.560 seconds\n",
      "[proc 1]sample: 64.553, forward: 410.305, backward: 52.927, update: 517.855\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.385383637060225\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3857840364083648\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4824800211712718\n",
      "[proc 1][Train](80000/100000) average loss: 0.4339318291500211\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.48226575775593516\n",
      "[proc 1][Train](80000/100000) average regularization: 0.00012083621266756381\n",
      "[proc 0][Train](80000/100000) average loss: 0.4340248971626163\n",
      "[proc 1][Train] 20000 steps take 1045.696 seconds\n",
      "[proc 1]sample: 65.152, forward: 410.928, backward: 52.114, update: 515.322\n",
      "[proc 0][Train](80000/100000) average regularization: 0.00012085921488978784\n",
      "[proc 0][Train] 20000 steps take 1045.707 seconds\n",
      "[proc 0]sample: 62.684, forward: 417.661, backward: 49.675, update: 512.887\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.38437231761664153\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4803336336120963\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.38401125945299863\n",
      "[proc 0][Train](100000/100000) average loss: 0.43235297554731367\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.48054979766011235\n",
      "[proc 0][Train](100000/100000) average regularization: 0.00012228715416495107\n",
      "[proc 0][Train] 20000 steps take 1043.291 seconds\n",
      "[proc 0]sample: 62.337, forward: 417.390, backward: 49.673, update: 510.742\n",
      "[proc 1][Train](100000/100000) average loss: 0.4322805284217\n",
      "[proc 1][Train](100000/100000) average regularization: 0.0001222590005672828\n",
      "[proc 1][Train] 20000 steps take 1043.301 seconds\n",
      "[proc 1]sample: 64.463, forward: 411.408, backward: 51.974, update: 514.287\n",
      "[1]Valid average MRR: 0.5321311507641732\n",
      "[1]Valid average MR: 61.461346752282836\n",
      "[1]Valid average HITS@1: 0.41401839885058256\n",
      "[1]Valid average HITS@3: 0.6073459215427253\n",
      "[1]Valid average HITS@10: 0.7421539320563542\n",
      "[proc 1]validation take 420.652 seconds:\n",
      "[0]Valid average MRR: 0.5306987907911145\n",
      "[0]Valid average MR: 62.68348586370322\n",
      "[0]Valid average HITS@1: 0.4119784006101215\n",
      "[0]Valid average HITS@3: 0.6069414937081222\n",
      "[0]Valid average HITS@10: 0.7413554774745329\n",
      "[proc 0]validation take 424.657 seconds:\n",
      "proc 0 takes 6127.222 seconds\n",
      "proc 1 takes 6127.222 seconds\n",
      "training takes 6127.735189676285 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_16\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5299729313494324\n",
      "Test average MR : 62.644429615203904\n",
      "Test average HITS@1 : 0.41152788086369735\n",
      "Test average HITS@3 : 0.6055363380703678\n",
      "Test average HITS@10 : 0.7400889981410488\n",
      "-----------------------------------------\n",
      "testing takes 426.422 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 18.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 18\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 6, 12, **18**\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.464 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.8947231962487101\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.9404542328007519\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5717325525728928\n",
      "[proc 1][Train](20000/100000) average loss: 0.7332278731614351\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5709611170214305\n",
      "[proc 1][Train](20000/100000) average regularization: 0.00030083042908345305\n",
      "[proc 0][Train](20000/100000) average loss: 0.7557076747342945\n",
      "[proc 1][Train] 20000 steps take 1069.010 seconds\n",
      "[proc 1]sample: 66.574, forward: 418.838, backward: 52.467, update: 526.862\n",
      "[proc 0][Train](20000/100000) average regularization: 0.0003192540522683885\n",
      "[proc 0][Train] 20000 steps take 1069.020 seconds\n",
      "[proc 0]sample: 65.231, forward: 426.589, backward: 52.221, update: 521.311\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4177720690101385\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.41715239266008136\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5152842239305377\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5153187747970224\n",
      "[proc 1][Train](40000/100000) average loss: 0.4665281464934349\n",
      "[proc 1][Train](40000/100000) average regularization: 0.0001387753227383655\n",
      "[proc 0][Train](40000/100000) average loss: 0.46623558374643326\n",
      "[proc 1][Train] 20000 steps take 1058.155 seconds\n",
      "[proc 1]sample: 64.805, forward: 411.585, backward: 51.847, update: 523.137\n",
      "[proc 0][Train](40000/100000) average regularization: 0.00013878508120324115\n",
      "[proc 0][Train] 20000 steps take 1058.161 seconds\n",
      "[proc 0]sample: 65.397, forward: 412.657, backward: 51.467, update: 523.148\n",
      "[1]Valid average MRR: 0.5013731332634773\n",
      "[1]Valid average MR: 63.219417528616276\n",
      "[1]Valid average HITS@1: 0.3791851937599161\n",
      "[1]Valid average HITS@3: 0.5767991992210109\n",
      "[1]Valid average HITS@10: 0.7233873768359697\n",
      "[proc 1]validation take 454.265 seconds:\n",
      "[0]Valid average MRR: 0.5005265604451742\n",
      "[0]Valid average MR: 64.67215844092172\n",
      "[0]Valid average HITS@1: 0.3780301792231846\n",
      "[0]Valid average HITS@3: 0.5760268562401264\n",
      "[0]Valid average HITS@10: 0.7222074685406112\n",
      "[proc 0]validation take 461.889 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.40651303433179853\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.4063877272635698\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5039132743269205\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5037974926173687\n",
      "[proc 1][Train](60000/100000) average loss: 0.45521315430402753\n",
      "[proc 0][Train](60000/100000) average loss: 0.4550926099076867\n",
      "[proc 1][Train](60000/100000) average regularization: 0.00014101447702487348\n",
      "[proc 0][Train](60000/100000) average regularization: 0.00014102278406571714\n",
      "[proc 1][Train] 20000 steps take 1522.796 seconds\n",
      "[proc 1]sample: 64.921, forward: 418.606, backward: 51.898, update: 523.588\n",
      "[proc 0][Train] 20000 steps take 1522.785 seconds\n",
      "[proc 0]sample: 63.732, forward: 418.021, backward: 51.411, update: 519.627\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.401093360966444\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.49814152106642723\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.40114178069382905\n",
      "[proc 0][Train](80000/100000) average loss: 0.4496174410104752\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4981979134500027\n",
      "[proc 0][Train](80000/100000) average regularization: 0.00014225436728884232\n",
      "[proc 0][Train] 20000 steps take 1047.420 seconds\n",
      "[proc 0]sample: 62.485, forward: 420.739, backward: 49.885, update: 512.951\n",
      "[proc 1][Train](80000/100000) average loss: 0.44966984703838825\n",
      "[proc 1][Train](80000/100000) average regularization: 0.00014223666555189993\n",
      "[proc 1][Train] 20000 steps take 1047.433 seconds\n",
      "[proc 1]sample: 64.582, forward: 412.491, backward: 51.744, update: 515.544\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.39770954300761224\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4944882156133652\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3977224917575717\n",
      "[proc 1][Train](100000/100000) average loss: 0.44609887921065094\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.494394905975461\n",
      "[proc 1][Train](100000/100000) average regularization: 0.0001430267587376875\n",
      "[proc 1][Train] 20000 steps take 1048.250 seconds\n",
      "[proc 1]sample: 64.288, forward: 412.446, backward: 51.712, update: 516.440\n",
      "[proc 0][Train](100000/100000) average loss: 0.44605869878679516\n",
      "[proc 0][Train](100000/100000) average regularization: 0.00014303509772216786\n",
      "[proc 0][Train] 20000 steps take 1048.265 seconds\n",
      "[proc 0]sample: 62.506, forward: 420.990, backward: 48.579, update: 514.503\n",
      "[0]Valid average MRR: 0.511921663585674\n",
      "[0]Valid average MR: 61.573572070599774\n",
      "[0]Valid average HITS@1: 0.38970828566759275\n",
      "[0]Valid average HITS@3: 0.5888693686332189\n",
      "[0]Valid average HITS@10: 0.7317133790924443\n",
      "[proc 0]validation take 457.180 seconds:\n",
      "[1]Valid average MRR: 0.5130283538431702\n",
      "[1]Valid average MR: 60.0321911791743\n",
      "[1]Valid average HITS@1: 0.39094833749838276\n",
      "[1]Valid average HITS@3: 0.5895565073506881\n",
      "[1]Valid average HITS@10: 0.7339963365723119\n",
      "[proc 1]validation take 462.112 seconds:\n",
      "proc 1 takes 6207.757 seconds\n",
      "proc 0 takes 6207.760 seconds\n",
      "training takes 6208.306072711945 seconds\n",
      "Save model to ckpts/TransE_l1_DRKG_17\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5108632774939161\n",
      "Test average MR : 61.56703970529154\n",
      "Test average HITS@1 : 0.3887233839721634\n",
      "Test average HITS@3 : 0.5871459991692599\n",
      "Test average HITS@10 : 0.7319858774181687\n",
      "-----------------------------------------\n",
      "testing takes 495.010 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name TransE_l1 \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 18.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
